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Highlights 

Highlights  of  GAO-1 1-239SP,  a  report  to 
congressional  committees 


NASA 

Assessments  of  Selected  Large-Scale  Projects 


Why  GAO  Did  This  Study 


What  GAO  Found 


GAO’s  work  has  shown  that  the 
National  Aeronautics  and  Space 
Administration’s  (NASA)  large-scale 
projects,  while  producing  ground¬ 
breaking  research  and  advancing 
our  understanding  of  the  universe, 
tend  to  cost  more  and  take  longer 
to  develop  than  planned,  and  are 
often  approved  without  evidence 
of  a  sound  business  case.  Although 
space  development  is  complex  and 
difficult  by  nature,  GAO  has  found 
that  inherent  risks  are  compounded 
by  the  need  for  better  management 
and  oversight  practices.  GAO  has 
designated  NASA’s  acquisition 
management  a  high  risk  area. 

This  report  provides  a  snapshot 
of  how  well  NASA  is  planning 
and  executing  its  acquisition  of 
selected  large-scale  projects.  It 
also  provides  observations  about 
the  performance  of  NASA’s  major 
projects  and  project  management, 
outlines  steps  NASA  is  taking  to 
improve  its  acquisitions,  identifies 
challenges  that  contribute  to  cost 
and  schedule  growth,  and  assesses 
21  NASA  projects,  each  with  an 
estimated  life-cycle  cost  of  over 
$250  million. 

GAO  is  not  making  any  new 
recommendations  in  this  report. 
Instead  GAO  is  issuing  another 
report  concurrently  (GAO-1 1- 
364R)  that  describes  in  more  detail 
some  of  the  issues  identified  in 
this  report,  such  as  transparency 
in  project  costs  and  lack  of  a 
consistent  design  metric,  and  will 
make  recommendations  to  address 
the  issues. 


GAO  assessed  21  NASA  projects  with  a  combined  life-cycle  cost  that  exceeds 
$68  billion.  Of  those  21  projects,  16  had  entered  the  implementation  phase 
where  cost  and  schedule  baselines  were  established.  Development  costs  for 
the  16  projects  had  an  average  growth  of  $94  million — or  14.6  percent — and 
schedules  grew  by  an  average  of  8  months.  The  total  increase  in  development 
costs  for  these  projects  was  $1.5  billion.  GAO  found  that  5  of  the  16  projects 
were  responsible  for  the  overwhelming  majority  of  this  increase.  The  issue  of 
cost  growth  is  more  significant  than  the  14.6  percent  average  would  indicate 
because  it  does  not  capture  the  cost  growth  that  occurred  before  several 
projects  reported  baselines  in  response  to  a  statutory  requirement  in  2005. 
Specifically,  the  13  projects  that  GAO  has  reviewed  over  the  past  3  years  that 
established  baselines  prior  to  2009  experienced  an  average  development  cost 
growth  of  almost  55  percent,  with  a  total  increase  in  development  costs  of 
almost  $2.5  billion  from  their  original  confirmation  baselines.  This  does  not 
reflect  considerable  cost  and  schedule  growth  that  will  likely  be  experienced 
by  NASA’s  largest  science  program — the  James  Webb  Space  Telescope  (JWST). 
Based  on  the  findings  of  the  independent  panel  that  recently  reviewed  the 
JWST  project  and  information  that  we  obtained  from  project  officials,  it  is 
likely  that  JWST  will  report  significant  cost  and  schedule  growth,  estimated  to 
be  $1.4  billion  or  more  and  up  to  15  months,  respectively. 

Many  of  the  projects  GAO  reviewed  for  this  report  experienced  challenges 
in  the  areas  of  technology,  design,  funding,  launch  vehicles,  development 
partner  performance,  parts,  and  contractor  management.  Reducing  the 
kinds  of  challenges  this  assessment  identifies  in  acquisition  programs  hinges 
on  developing  a  sound  business  case  for  a  project.  The  development  and 
execution  of  a  knowledge-based  business  case  for  these  projects  can  provide 
early  recognition  of  challenges,  allow  managers  to  take  corrective  action, 
and  place  needed  and  justifiable  projects  in  a  better  position  to  succeed.  The 
inherent  complexity  of  space  development  programs  should  not  preclude 
NASA  from  achieving  what  it  promises  when  requesting  and  receiving  funds. 

In  response  to  GAO’s  designation  of  NASA’s  acquisition  management  as  a 
high  risk  area,  NASA  has  developed  a  corrective  action  plan  to  improve 
the  effectiveness  of  acquisition  project  management.  The  plan  identifies 
five  areas  for  improvement,  each  of  which  contains  targets  and  goals  to 
measure  improvement.  As  part  of  this  initiative,  the  agency  is  continuing  its 
implementation  of  a  new  cost  estimation  tool,  the  Joint  Cost  and  Schedule 
Confidence  Level,  to  help  project  officials  with  management,  cost  and  schedule 
estimating,  and  maintenance  of  adequate  levels  of  reserves. 


View  GAO-1 1-239SP  or  key  components. 
For  more  information,  contact 
Cristina  Chaplain  at  (202)  512-4841  or 
chaplainc@gao.gov. 
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United  States  Government  Accountability  Office 
Washington,  DC  20548 


March  3,2011 
Congressional  Committees 

We  are  pleased  to  present  GAO’s  third  annual  assessment  of  selected  large- 
scale  National  Aeronautics  and  Space  Administration  (NASA)  projects. 

This  report  provides  a  snapshot  of  NASA’s  planning  and  execution  of  major 
acquisitions — a  topic  that  is  on  GAO’s  high  risk  list. 

This  past  year  has  been  one  of  turmoil  for  NASA.  The  proposed  cancellation 
of  the  Constellation  program — the  agency’s  largest  program — has  left 
NASA’s  human  space  flight  program  in  a  state  of  flux.  Its  future  work  in  this 
area  depends  on  how  budget  issues  and  direction  are  resolved  between 
the  Congress  and  the  Administration.  While  NASA  continued  to  work 
toward  the  program  of  record  for  Constellation,  its  focus  has  now  turned 
to  prioritizing  work  that  can  be  transitioned  to  the  new  path  for  human 
space  flight  set  out  in  the  NASA  Authorization  Act  of  2010  while  continuing 
to  comply  with  the  requirements  of  its  fiscal  year  2010  appropriations. 
Additionally,  funding  constraints  due  to  the  delayed  retirement  of  the 
shuttle  fleet,  the  plan  to  utilize  the  International  Space  Station  at  least  4 
years  longer  than  anticipated,  and  expected  overruns  in  major  projects, 
such  as  the  James  Webb  Space  Telescope  and  the  Mars  Science  Lab,  will 
affect  NASA’s  plans  for  funding  new  projects  for  years  to  come.  This 
environment,  coupled  with  a  constrained  budgetary  outlook,  heightens 
the  importance  of  efficient  and  effective  project  management  to  maximize 
results.  Furthermore,  NASA  needs  to  be  equipped  with  the  knowledge  to 
make  hard  choices  among  competing  priorities  within  the  agency. 

We  recently  issued  an  update  to  our  high  risk  report  where  we  highlighted 
efforts  NASA  continues  to  make  to  improve  its  management  of  major 
projects.  For  example,  the  agency  has  continued  to  implement  initiatives 
aimed  at  strengthening  its  cost  and  schedule  estimating  processes.  These 
initiatives,  as  well  as  other  efforts,  are  intended  to  provide  key  decision¬ 
makers  with  increased  knowledge  to  make  informed  decisions  before  a 
project  starts  and  to  maintain  disciplined  management  and  oversight  once 
it  begins.  Increased  discipline  and  oversight,  however,  will  require  that 
senior  NASA  leaders  have  the  will  to  terminate  or  reshape  projects  that  do 
not  measure  up,  hold  appropriate  parties  accountable  for  poor  outcomes, 
and  recognize  and  reward  good  management  and  good  decisions.  NASA 
continues  to  take  positive  steps,  but  it  will  still  be  some  time  before  the 
impact  of  its  efforts  can  be  measured. 
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The  NASA  portfolio  of  major  projects  ranges  from  robotic  probes  designed 
to  explore  the  Martian  surface,  to  satellites  equipped  with  advanced  sensors 
to  study  the  earth,  to  telescopes  intended  to  explore  the  universe.  Some 
of  these  missions  have  literally  changed  the  way  we  view  our  planet  and 
the  universe.  For  example,  the  Kepler  mission  recently  identified  the  first 
Earth-size  planet  candidates  in  a  habitable  zone  where  liquid  water  could 
exist  on  the  planet’s  surface.  In  many  cases,  NASA’s  projects  are  expected  to 
incorporate  new  and  sophisticated  technologies  that  must  operate  in  harsh, 
distant  environments. 

Although  space  development  programs  are  complex  and  difficult  by 
nature,  our  work  consistently  finds  that  inherent  risks  of  NASA’s  complex 
development  projects  are  heightened  by  the  induced  risks  of  less  than 
adequate  management  and  oversight  practices.  In  this  year’s  report,  our 
work  continues  to  show  that  NASA’s  major  projects  are  frequently  approved 
without  evidence  of  a  sound  business  case  that  ensures  a  match  between 
requirements  and  reasonably  expected  resources.  As  a  result,  the  projects 
cost  more  and  take  longer  to  develop  than  planned.  We  found  that  NASA 
frequently  exceeded  its  acquisition  cost  and  schedule  estimates,  even 
when  those  estimates  were  relatively  new.  In  the  last  3  years,  12  out  of  the 
13  projects  that  have  been  in  development  for  several  years  significantly 
exceeded  their  cost  and/or  schedule  baseline  estimates.  In  today’s  fiscal 
environment,  it  is  clear  that  this  condition  cannot  be  sustained. 

We  believe  that  this  report  can  provide  insights  that  will  help  NASA  place 
programs  in  a  better  position  to  succeed,  and  help  the  agency  maximize 
its  investments.  Our  work  has  shown  that  curbing  the  induced  challenges 
that  can  lead  to  cost  and  schedule  growth  hinges  on  developing  a  sound 
business  case  that  includes  firm  requirements,  mature  technologies,  a 
knowledge-based  acquisition  strategy,  realistic  cost  estimates,  and  sufficient 
funding.  Consistent  adoption  of  such  practices  can  improve  results  and  may 
help  ease  the  budgetary  pressures  NASA  is  likely  to  continue  to  face  over 
time. 


Gene  L.  Dodaro 
Comptroller  General 
of  the  United  States 
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Washington,  DC  20548 


March  3,2011 
Congressional  Committees 

This  is  GAO’s  third  annual  assessment  of  the  National  Aeronautics  and 
Space  Administration’s  (NASA)  large-scale  projects.  This  report  provides 
a  snapshot  of  how  well  NASA  is  planning  and  executing  its  major 
acquisitions — an  area  that  has  been  on  GAO’s  high  risk  list  since  1990.  Over 
the  past  year,  NASA  has  again  showed  that  its  projects  produce  ground¬ 
breaking  research  and  advance  our  understanding  of  the  universe.  For 
example,  the  Kepler  spacecraft  has  discovered  the  first  confirmed  planetary 
system  with  more  than  one  planet  transiting  the  same  star.  Unfortunately, 
over  the  past  year,  NASA  has  also  experienced  much  turmoil  and  many 
cost  increases  in  several  of  its  major  projects.  For  example,  the  proposed 
cancellation  of  the  Constellation  Program,  after  spending  over  $11  billion 
since  2006,  caused  uncertainty  in  NASA’s  human  spaceflight  program.  More 
recently,  an  independent  panel  concluded  that  the  James  Webb  Space 
Telescope  project  will  require  additional  funding  of  $1.4  billion  or  more  and 
a  launch  delay  of  15  months.  In  the  past  2  years,  we  reported  that  11  out 
of  17  NASA  projects  experienced  significant  cost  and/or  schedule  growth 
from  baselines  established  only  2  or  3  years  earlier.1  Such  issues  continue 
to  impact  NASA’s  ability  to  conduct  its  ground-breaking  work  in  an  efficient 
and  effective  manner. 

NASA  has  taken  steps  over  recent  years  to  help  improve  its  acquisition 
management  through  several  initiatives  aimed  at  improving  cost  estimating 
and  management  oversight.  While  the  overall  outcomes  of  these  efforts  will 
take  time  to  become  apparent,  NASA  officials  indicate  that  they  continue  to 
be  committed  to  the  initiatives  with  the  goal  of  improving  performance. 

The  Congress  has  expressed  concern  about  NASA’s  performance  and  has 
identified  the  need  to  standardize  the  reporting  of  cost,  schedule,  and 
content  for  NASA  research  and  development  projects.  In  2005,  the  Congress 
required  NASA  to  report  cost  and  schedule  baselines — benchmarks  against 
which  changes  can  be  measured — for  all  NASA  programs  and  projects  with 
estimated  life-cycle  costs  of  at  least  $250  million  that  have  been  approved 
to  proceed  to  the  development  stage,  known  as  implementation,  in  which 


1  GAO,  NASA:  Assessments  of  Selected  Large-Scale  Projects,  GAO-09-306SP  (Washington, 
D.C.:  Mar.  2,  2009)  and  GAO,  NASA:  Assessments  of  Selected  Large-Scale  Projects, 
GAO-10-227SP  (Washington,  D.C.:  Feb.  1,  2010). 
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components  begin  to  take  physical  form.2  It  also  required  that  NASA 
report  to  Congress  when  development  cost  is  likely  to  exceed  the  baseline 
estimate  by  15  percent  or  more,  or  when  a  milestone  is  likely  to  be  delayed 
beyond  the  baseline  estimate  by  6  months  or  more.3  In  response,  NASA 
began  to  report  new  cost  and  schedule  baselines  in  2006  and  has  been  using 
them  as  the  basis  for  annual  project  performance  reports  to  the  Congress 
provided  in  its  budget  submission  each  year. 

The  explanatory  statement  of  the  House  Committee  on  Appropriations 
accompanying  the  Omnibus  Appropriations  Act,  2009  directed  GAO  to 
prepare  project  status  reports  on  selected  large-scale  NASA  programs, 
projects,  or  activities.4 5  This  report  responds  to  that  mandate.  Specifically, 
we  assess  (1)  performance  of  NASA’s  major  projects  and  the  agency’s 
management  of  those  projects  during  development,  (2)  knowledge  attained 
by  key  junctures  in  the  acquisition  process,  (3)  other  challenges  that 
can  affect  project  execution,  (4)  NASA’s  continued  efforts  to  improve  its 
acquisition  management,  and  (5)  21  NASA  projects,  each  with  an  estimated 
life-cycle  cost  over  $250  million.6  In  doing  so,  the  report  expands  on  the 
importance  of  providing  decision-makers  with  an  independent,  knowledge- 
based  assessment  of  individual  systems  that  identifies  potential  risks  and 
allows  them  to  take  actions  to  put  projects  that  are  early  in  the  development 
cycle  in  a  better  position  to  succeed. 

Our  approach  included  an  examination  of  the  current  phase  of  a  project’s 
development  and  how  each  project  was  advancing.6  NASA  provided 
updated  cost  and  schedule  data  as  of  February  2011  for  16  of  the  21 
projects.  We  reviewed  and  compared  that  data  to  previously  established 
cost  and  schedule  baselines.  We  assessed  each  project’s  cost  and  schedule 


2  National  Aeronautics  and  Space  Administration  Authorization  Act  of  2005,  Pub.  L.  No.  109- 
155,  §103;  42  U.S.C.  §  16613(b). 

3  42  U.S.C.  §  16613(d). 

4  See  Explanatory  Statement  accompanying  the  Omnibus  Appropriations  Act,  2009,  Pub.  L. 
No.  111-8. 

5  Each  assessment  is  presented  in  a  2-page  summary  that  analyzes  the  project’s  cost  and 
schedule  status  and  project  challenges  we  identified  with  the  objective  to  identify  risks  that, 
if  mitigated,  could  put  NASA  in  a  better  position  to  succeed. 

6  Each  project  we  reviewed  was  in  either  the  formulation  phase  or  the  implementation  phase 
of  the  project  life  cycle.  In  the  formulation  phase,  the  project  defines  requirements — what 
the  project  is  being  designed  to  do — matures  technology,  establishes  a  schedule,  estimates 
costs,  and  produces  a  plan  for  implementation.  In  the  implementation  phase,  the  project 
carries  out  these  plans,  performing  final  design  and  fabrication  as  well  as  testing  components 
and  system  assembly,  integrating  these  components  and  testing  how  they  work  together, 
and  launching  the  project.  This  phase  also  includes  the  period  from  project  launch  through 
mission  completion. 
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and  characterized  growth  either  as  significant  if  it  exceeded  the  thresholds 
that  trigger  cost  or  schedule  growth  reporting  to  the  Congress  under  the 
law.* * * * * * 7  In  addition,  NASA  provided  cost  and  schedule  information  from 
previously  reported  projects  that  we  used  for  historical  analysis.  We 
assessed  technology  maturity  and  design  stability  using  GAO’s  established 
criteria  for  knowledge-based  acquisitions  and  other  GAO  work  on 
system  acquisitions.8  Additionally,  as  a  result  of  our  analysis  based  on 
interviews  with  project  officials  and  information  provided  by  the  projects, 
we  identified  other  challenges — funding,  launch  vehicles,  development 
partner  performance,  parts,  and  contractor  management — that  can  affect 
project  outcomes.  This  list  of  challenges  is  not  exhaustive  and  we  believe 
these  challenges  will  evolve,  as  they  have  from  previous  years,  as  we 
continue  this  work  into  the  future.  We  took  appropriate  steps  to  address 
data  reliability.  The  individual  project  offices  were  given  an  opportunity  to 
provide  comments  and  technical  clarifications  on  our  assessments  prior  to 
their  inclusion  in  the  final  product,  which  were  incorporated  as  appropriate. 
Appendix  II  contains  detailed  information  on  our  scope  and  methodology. 

We  conducted  this  performance  audit  from  March  2010  to  February  2011  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions  based 
on  our  audit  objectives.  We  are  not  making  recommendations  in  this  report. 


Background 


A  Sound  Business  Case  The  development  and  execution  of  a  knowledge-based  business  case  for 

Underpins  Successful  NASA’s  projects  can  provide  early  recognition  of  challenges,  allow 

Acquisition  Outcomes  managers  to  take  corrective  action,  and  place  needed  and  justifiable 

projects  in  a  better  position  to  succeed.  Our  studies  of  best  practice 

organizations  show  the  risks  inherent  in  NASA’s  work  can  be  mitigated  by 

developing  a  solid,  executable  business  case  before  committing  resources 


7  NASA  is  required  to  report  to  Congress  if  development  cost  of  a  program  is  likely  to  exceed 
the  baseline  estimate  by  15  percent  or  more,  or  if  a  milestone  is  likely  to  be  delayed  by  6 
months  or  more.  42  U.S.C.  §  16613(d). 

8  GAO,  Best  Practices:  Using  a  Knowledge-Based  Approach  to  Improve  Weapon 
Acquisition,  GAO-04-386SP  (Washington,  D.C.:  Jan.  2004). 
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to  a  new  product  development.9  In  its  simplest  form,  this  is  evidence  that 
(1)  the  customer’s  needs  are  valid  and  can  best  be  met  with  the  chosen 
concept  and  that  (2)  the  chosen  concept  can  be  developed  and  produced 
within  existing  resources — that  is,  proven  technologies,  design  knowledge, 
adequate  funding,  adequate  time,  and  adequate  workforce  to  deliver  the 
product  when  needed.  A  program  should  not  be  approved  to  go  forward 
into  product  development  unless  a  sound  business  case  can  be  made.  If  the 
business  case  measures  up,  the  organization  commits  to  the  development  of 
the  product,  including  making  the  financial  investment.  Our  best  practice 
work  has  shown  that  developing  business  cases  based  on  matching 
requirements  to  resources  before  program  start  leads  to  more  predictable 
program  outcomes — that  is,  programs  are  more  likely  to  be  successfully 
completed  within  cost  and  schedule  estimates  and  deliver  anticipated 
system  performance.10 

At  the  heart  of  a  business  case  is  a  knowledge-based  approach  to  product 
development  that  is  a  best  practice  among  leading  commercial  firms.  Those 
firms  have  created  an  environment  and  adopted  practices  that  put  their 
program  managers  in  a  good  position  to  succeed  in  meeting  expectations. 

A  knowledge-based  approach  requires  that  managers  demonstrate  high 
levels  of  knowledge  as  the  program  proceeds  from  technology  development 
to  system  development  and,  finally,  production.  In  essence,  knowledge 
supplants  risk  over  time.  This  building  of  knowledge  can  be  described  over 
the  course  of  a  program  as  follows: 

•  When  a  project  begins  development,  the  customer’s  needs  should  match 
the  developer’s  available  resources — mature  technologies,  time,  and 
funding.  An  indication  of  this  match  is  the  demonstrated  maturity  of  the 
technologies  required  to  meet  customer  needs — referred  to  as  critical 
technologies.  If  the  project  is  relying  on  heritage — or  pre-existing — 
technology,  that  technology  must  be  in  appropriate  form,  fit,  and 
function  to  address  the  customer’s  needs  within  available  resources. 

The  project  will  normally  enter  development  after  completing  the 
preliminary  design  review,  at  which  time  a  business  case  should  be  in 
hand. 


9  GAO,  Defense  Acquisitions:  Key  Decisions  to  Be  Made  on  Future  Combat  System, 
GAO-07-376  (Washington,  D.C.:  Mar.  15,  2007);  Defense  Acquisitions:  Improved  Business 
Case  Key  for  Future  Combat  System’s  Success,  GAO-06-564T  (Washington,  D.C.:  Apr.  4, 
2006);  NASA:  Implementing  a  Knowledge-Based  Acquisition  Framework  Could  Lead  to 
Better  Investment  Decisions  and  Project  Outcomes,  GAO-06-218  (Washington,  D.C.:  Dec. 
21,  2005);  and  NASA’s  Space  Vision:  Business  Case  for  Prometheus  1  Needed  to  Ensure 
Requirements  Match  Available  Resources,  GAO-05-242  (Washington,  D.C.:  Feb.  28,  2005). 

10  GAO-05-242. 
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•  Then,  about  midway  through  the  product’s  development,  its  design 
should  be  stable  and  demonstrate  it  is  capable  of  meeting  performance 
requirements.  The  critical  design  review  takes  place  at  that  point  in  time 
because  it  generally  signifies  when  the  program  is  ready  to  start  building 
production-representative  prototypes.  If  design  stability  is  not  achieved, 
but  a  product  development  continues,  costly  re-designs  to  address 
changes  to  project  requirements  and  unforeseen  challenges  can  occur. 
By  the  critical  design  review,  the  design  should  be  stable  and  capable  of 
meeting  performance  requirements. 

•  Finally,  by  the  time  of  the  production  decision,  the  product  must  be 
shown  to  be  producible  within  cost,  schedule,  and  quality  targets  and 
have  demonstrated  its  reliability,  and  the  design  must  demonstrate 
that  it  performs  as  needed  through  realistic  system-level  testing.  Lack 
of  testing  increases  the  possibility  that  project  managers  will  not  have 
information  that  could  help  avoid  costly  system  failures  in  late  stages  of 
development  or  during  system  operations. 

Our  best  practices  work  has  identified  numerous  other  actions  that  can 
be  taken  to  increase  the  likelihood  that  a  program  can  be  successfully 
executed  once  that  business  case  is  established.  These  include  ensuring 
cost  estimates  are  complete,  accurate  and  updated  regularly,  and  holding 
suppliers  accountable  through  such  activities  as  regular  supplier  audits 
and  performance  evaluations  of  quality  and  delivery.  Moreover,  we  have 
recommended  using  metrics  and  controls  throughout  the  life  cycle  to  gauge 
when  the  requisite  level  of  knowledge  has  been  attained  and  when  to  direct 
decision  makers  to  consider  criteria  before  advancing  a  program  to  the  next 
level  and  making  additional  investments. 

NASA  Life  Cycle  for  Flight  NASA  life  cycle  for  flight  system  is  defined  by  two  phases — formulation* 11 
Systems  and  implementation12 — and  several  key  decision  points.  See  figure  1.  These 


11  NASA  defines  formulation  as  the  identification  of  how  the  program  or  project  supports 
the  agency’s  strategic  needs,  goals,  and  objectives;  the  assessment  of  feasibility,  technology 
and  concepts;  risk  assessment,  team  building,  development  of  operations  concepts  and 
acquisition  strategies;  establishment  of  high-level  requirements  and  success  criteria;  the 
preparation  of  plans,  budgets,  and  schedules  essential  to  the  success  of  a  program  or 
project;  and  the  establishment  of  control  systems  to  ensure  performance  to  those  plans 
and  alignment  with  current  agency  strategies.  NASA  Interim  Directive  (NID)  NM  7120-81 
for  NASA  Procedural  Requirements  (NPR)  7120.5D,  paragraph  1.2.1(a)  (Sept.  22,  2009) 
(Hereinafter  cited  as  NID  for  NPR  7120.5D  (Sept.  22,  2009)). 

12  The  implementation  phase  is  defined  as  the  execution  of  approved  plans  for  the 
development  and  operation  of  the  program/project,  and  the  use  of  control  systems  to  ensure 
performance  to  approved  plans  and  continued  alignment  with  the  agency’s  strategic  needs, 
goals,  and  objectives.  NID  for  NPR  7120.5D,  paragraph  1.2.1(c)  (Sept.  22,  2009). 
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phases  are  then  further  divided  into  incremental  pieces:  Phase  A  through 
Phase  F. 


Figure  1 :  NASA’s  Life  Cycle  for  Flight  Systems 


NASA’s  life  cycle  for  flight  systems  and  ground  support  projects 


Formulation  Pre-NAR  nar  Implementation 
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▼  Pre-NAR  =  preliminary  non  advocate  review 
'W  NAR  =  non  advocate  review 
KDP  =  key  decision  point 
Technical  reviews 
|  SDR  =  system  definition  review 
|  PDR  =  preliminary  design  review 
|  CDR  =  critical  design  review 

Source:  NASA  data  and  GAO  analysis. 

Project  formulation  consists  of  Phases  A  and  B,  during  which  time 
the  projects  develop  and  define  the  project  requirements  and  cost/ 
schedule  basis  and  design  for  implementation,  including  developing  an 
acquisition  strategy.  During  the  end  of  the  formulation  phase,  leading 
up  to  the  preliminary  design  review  (PDR)13  and  non-advocate  review 
(NAR),14  the  project  team  completes  its  preliminary  design  and  technology 


13  According  to  NID  for  NPR  7120.5D,  Table  2-7  (Sept.  22,  2009),  the  PDR  demonstrates  that 
the  preliminary  design  meets  all  system  requirements  with  acceptable  risk  and  within  the 
cost  and  schedule  constraints  and  establishes  the  basis  for  proceeding  with  detailed  design. 
It  shows  that  the  correct  design  option  has  been  selected,  interfaces  have  been  identified, 
and  verification  methods  have  been  described.  Full  baseline  cost  and  schedules,  as  well  as 
risk  assessments,  management  systems,  and  metrics  are  presented. 

14  According  to  NID  for  NPR  7120.5D,  Appendix  A  (Sept.  22,  2009),  a  NAR  is  comprised 
of  the  analysis  of  a  proposed  program  or  project  by  a  (non-advocate)  team  composed  of 
management,  technical,  and  resources  experts  (personnel)  from  outside  the  advocacy  chain 
of  the  proposed  program  or  project.  It  provides  agency  management  with  an  independent 
assessment  of  the  readiness  of  the  program/project  to  proceed  into  implementation. 
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development.  NASA  Interim  Directive  NM  7120-81  for  NASA  Procedural 
Requirements  7120.5D,  NASA  Space  Flight  Program  and  Project 
Management  Requirements,  specifies  that  during  formulation  the  project 
should  complete  development  of  mission-critical  or  enabling  technology. 

As  needed,  projects  are  required  to  demonstrate  evidence  of  technology 
maturity  (i.e.,  component  and/or  breadboard  validation  in  the  relevant 
environment)  and  document  the  information  in  a  technology  readiness 
assessment  report.  The  project  must  also  develop,  document,  and  maintain 
a  project  management  baseline16  that  includes  the  integrated  master 
schedule  and  baseline  life-cycle  cost  estimate.  The  formulation  phase  is 
intended  to  culminate  in  a  confirmation  review  at  which  time  cost  and 
schedule  baselines  are  confirmed  and  project  progress  hence  forth  is 
measured  against  these  baselines. 

After  a  project  is  confirmed,  it  begins  implementation,  consisting  of 
phases  C,  D,  E,  and  F.  During  phase  C,  the  project  performs  final  design 
and  fabrication  as  well  as  testing  of  components.  In  phase  D,  the  project 
performs  system  assembly,  integration,  test,  and  launch  activities.  Phases 
E  and  F  consist  of  operations  and  sustainment  and  project  closeout.  A 
second  design  review,  the  critical  design  review  (CDR),16  is  held  in  the 
implementation  phase  during  the  latter  half  of  phase  C.  The  purpose  of 
the  CDR  is  to  demonstrate  that  the  maturity  of  the  design  is  appropriate  to 
support  proceeding  with  full-scale  fabrication,  assembly,  integration,  and 
test.  After  CDR  and  the  system  integration  review,17  the  project  must  be 
approved  before  continuing  into  the  next  phase. 

NASA  Projects  Reviewed 
in  GAO  Annual 
Assessments 


The  portfolio  of  projects  we  reviewed  has  evolved  and  grown  in  each  of  the 
last  3  years.  Once  a  project  launches,  we  will  no  longer  include  a  2-page 
summary  in  our  annual  report.  However,  we  do  maintain  and  continually 
assess  historical  cost,  schedule,  and  performance  information  collected 


15  The  management  baseline  is  the  integrated  set  of  requirements,  cost,  schedule,  technical 
content,  and  associated  joint  confidence  level  that  forms  the  foundation  for  program  or 
project  execution  and  reporting  done  as  part  of  NASA’s  performance  assessment  and 
governance  process.  NID  for  NPR  7120.5D,  paragraph  2.1. 8.2  and  Appendix  A  (Sept.  22, 

2009). 

16  According  to  NID  for  NPR  7120.5D,  Table  2-7  (Sept.  22,  2009),  the  CDR  demonstrates  that 
the  maturity  of  the  design  is  appropriate  to  support  proceeding  with  full-scale  fabrication, 
assembly,  integration,  and  test,  and  that  the  technical  effort  is  on  track  to  complete  the 
flight  and  ground  system  development  and  mission  operations  in  order  to  meet  mission 
performance  requirements  within  the  identified  cost  and  schedule  constraints.  Progress 
against  management  plans,  budget,  and  schedule,  as  well  as  risk  assessments  are  presented. 

17  The  system  integration  review  evaluates  the  readiness  of  the  project  to  start  flight  system 
assembly,  test,  and  launch  operations.  This  review  takes  place  after  the  CDR  and  just  prior  to 
the  beginning  of  phase  D,  where  test  and  integration  activities  occur.  NID  for  NPR  7120.5D, 
Table  2-7  and  paragraph  4.6.1  (Sept.  22,  2009). 
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from  these  projects  during  the  course  of  our  reviews.  As  NASA  determines 
that  a  project  will  have  a  life-cycle  cost  estimate  of  more  than  $250  million, 
we  will  then  include  that  project  in  the  next  review.  See  table  1  for  a  history 
of  the  projects  we  have  reviewed  over  the  past  3  years. 


Table  1 :  Selected  Major  NASA  Projects  Reviewed  in  GAO  Annual  Assessments 


2009 

2010 

2011 

Projects  in  Formulation 

Ares  1 

Ares  1 

Ares  1 

GPM 

GPM 

ICESat-2 

JWST 

LDCM 

Orion 

LDCM 

Orion 

SMAP 

Orion 

SPP 

Projects  in  Implementation 

Aquarius 

Aquarius 

Aquarius 

Dawn* 

Glory 

Glory 

GLAST* 

GRAIL 

GPM 

Glory 

Herschel* 

GRAIL 

Herschel 

Juno 

Juno 

Kepler 

JWST 

JWST 

LRO 

Kepler* 

LADEE 

MSL 

LRO* 

LDCM 

NPP 

MMS 

MAVEN 

OCO** 

MSL 

MSL 

SDO 

NPP 

MMS 

SOFIA 

RBSP 

NPP 

WISE 

SDO* 

OCO-2 

SOFIA 

RBSP 

WISE* 

SOFIA 

TDRS 

Replenishment 

Source:  GAO  Analysis  of  NASA  data. 

'NASA  projects  that  have  launched  and  proceeded  into  operations. 
“NASA  project  that  launched  but  failed  to  reach  orbit. 


Observations  on 
NASA’s  Portfolio  of 
Major  Projects 


We  assessed  21  large-scale  NASA  projects  in  this  review.  We  based  the 
majority  of  our  cost  and  schedule  analysis  on  the  16  projects  that  are 
currently  in  the  implementation  phase  of  the  project  life  cycle.  We  also 
analyzed  historical  data  from  projects  that  were  a  part  of  our  previous 
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reviews.  We  found  that  5  of  the  16  projects  currently  in  implementation 
experienced  significant  cost  and/or  schedule  growth  from  their  baselines.18 
The  remaining  11  projects  established  baselines  in  fiscal  year  2009  or  later 
and  have  reported  little  or  no  deviations  from  their  cost  and  schedule 
baselines.  Three  of  the  1 1  projects  that  had  been  in  formulation  for  most  of 
our  review  were  confirmed  late  in  2010,  and  their  baselines  were  reported 
for  the  first  time  in  the  NASA’s  fiscal  year  2012  budget  submission.  The 
remaining  five  projects  were  in  the  formulation  phase  where  cost  and 
schedule  baselines  have  yet  to  be  established.19  See  figure  2  for  a  summary 
of  these  projects. 


Figure  2:  Summary  of  Projects  Assessed  by  Phase  of  the  NASA  Project  Life  Cycle 


18  For  purposes  of  our  analysis,  cost  or  schedule  growth  is  significant  if  it  exceeds  the 
thresholds  that  trigger  reporting  to  Congress  under  the  law.  The  thresholds  are  development 
cost  growth  of  15  percent  or  more  from  the  baseline  cost  estimate  or  a  milestone  delay  of  6 
months  or  more  beyond  the  baseline  schedule  estimate.  42  U.S.C.  §  16613(d). 

19  NASA  did  not  provide  formal  cost  and  schedule  baselines  for  the  projects  in  formulation, 
citing  that  the  estimates  are  preliminary.  Baselines  are  established  when  the  project 
transitions  to  implementation. 
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Development  costs  for  the  16  projects  currently  in  implementation  had  an 
average  development  cost  growth  of  $94.3  million — or  14.6  percent — and 
schedule  growth  of  8  months  from  their  baselines.  The  total  increase  in 
development  costs  for  the  16  projects  in  implementation  was  over  $1.5 
billion.  The  five  projects  that  established  baselines  before  fiscal  year  2009 
were  responsible  for  the  overwhelming  majority  of  this  increase.  All  five 
projects  have  exceeded  cost  and  schedule  thresholds  set  by  the  Congress. 
Two  projects — Glory  and  MSL — were  re-baselined,  but  to  gain  a  more 
accurate  picture  of  cost  and  schedule  growth,  we  used  their  original 
baselines  for  our  analysis.  See  table  2. 


Table  2:  Cost  and  Schedule  Growth  of  Selected  NASA  Projects  Currently  in  the  Implementation  Phase  (dollars  in  millions) 


Project 

Baseline 

(FY) 

Development 
cost  growth 

Percentage 
cost  growth 

Launch 
delay  (months) 

NPP 

2007 

$187.1 

31.6 

42 

SOFIA 

2007 

$208.9 

22.7 

12 

Aquarius 

2008 

$34.6 

18.0 

23 

Glory3 

2008 

$168.7 

99.9 

27 

MSLb 

2008 

$833.4 

86.0 

26 

GRAIL 

2009 

$0.0 

0.0 

0 

Juno 

2009 

$0.0 

0.0 

0 

JWST 

2009 

$129.8 

5.0 

0 

RBSP 

2009 

$0.1 

0.0 

0 

GPM 

2010 

-$40.4 

-7.3 

0 

LDCM 

2010 

$4.2 

0.7 

0 

MMS 

2010 

$0.0 

0.0 

0 

TDRS  Replenishment 

2010 

-$17.2 

-8.2 

0 

LADEE 

2011 

$0.0 

0.0 

0 

MAVEN 

2011 

$0.0 

0.0 

0 

OCO-2 

2011 

$0.0 

0.0 

0 

Average 

$94.3 

14.6% 

8 

Total  development  cost 

$1,509.9 

Source:  GAO  analysis  of  NASA  data. 

Note:  Shading  indicates  projects  that  exceeded  their  cost  and/or  schedule  thresholds. 

aG lory  established  a  new  baseline  in  fiscal  year  2009  after  being  reauthorized  by  Congress. 
bM  SL  established  a  new  baseline  in  fiscal  year  2010  after  being  reauthorized  by  Congress. 

This  table  does  not  reflect  considerable  cost  and  schedule  growth  that 
will  likely  be  experienced  by  NASA’s  largest  science  program — the  James 
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Webb  Space  Telescope.  Based  on  the  findings  of  the  independent  panel 
that  recently  reviewed  the  JWST  project  and  information  we  obtained 
from  projects  officials,  it  is  likely  that  JWST  will  report  significant  cost  and 
schedule  growth,  currently  estimated  to  be  $1.4  billion  or  more  and  up  to  15 
months,  respectively. 

Table  2  also  includes  information  from  11  projects  that  were  all  confirmed 
in  the  last  two  years  and  have  not  reported  significant  cost  or  schedule 
growth.  Many  of  these  projects  are  entering,  or  have  recently  entered,  the 
test  and  integration  phase  where  cost  and  schedule  growth  is  typically 
realized.  Specifically,  seven  projects  plan  to  have  their  system  integration 
review  in  fiscal  year  2011  or  2012.  Importantly,  many  of  these  projects  have 
experienced  similar  challenges  as  the  older  projects  that  have  reported  cost 
and/or  schedule  growth,  such  as  issues  with  maturing  technology  and  not 
meeting  design  criteria. 

As  previously  stated,  the  Glory  and  MSL  projects  both  sought 
reauthorization  from  Congress  because  of  development  cost  growth 
in  excess  of  30  percent  despite  having  baselines  established  in  2008.20 
Congress  reauthorized  the  Glory  project,  and  new  cost  and  schedule 
baselines  were  established  in  fiscal  year  2009, 21  after  the  project 
experienced  a  53  percent  cost  growth  and  6-month  launch  delay  from 
original  baseline  estimates  established  in  fiscal  year  2008.  Although  Glory’s 
development  costs  have  increased  by  almost  31  percent  from  the  new 
baseline  established  in  2009,  Glory  is  scheduled  to  launch  in  March  2011 
before  a  second  reauthorization  would  need  to  be  sought.  Similarly,  MSL 
was  reauthorized  by  the  Congress  and  NASA  established  new  cost  and 
schedule  baselines  early  in  fiscal  year  2010  after  reporting  a  68  percent 
growth  in  cost  and  a  26  month  schedule  delay  from  its  original  baselines 
established  in  fiscal  year  2008. 

The  issue  of  cost  growth  is  more  significant  than  the  14.6  percent  average 
identified  in  table  2  would  indicate.  For  some  of  these  projects,  the  14.6 
percent  average  cost  growth  identified  does  not  capture  the  cost  growth 
that  occurred  from  the  time  these  projects  initially  established  baselines 
at  confirmation  to  the  time  that  they  reported  baselines  subsequently 
established  in  response  to  the  statutory  requirement.  Using  that  data,  we 
found  that  NASA’s  major  projects  have  experienced  an  average  development 


20  If  development  cost  of  a  program  will  exceed  the  baseline  estimate  by  more  than  30 
percent,  then  NASA  is  required  to  seek  reauthorization  from  Congress  in  order  to  continue 
the  program.  If  the  program  is  reauthorized,  NASA  is  required  to  establish  new  cost  and 
schedule  baselines.  42  U.S.C.  §  16613(e). 

21  42  U.S.C.  §  16613(e). 
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cost  growth  of  almost  55  percent,  with  the  total  increase  in  development 
costs  of  almost  $2.5  billion  from  their  baselines  established  at  confirmation. 
To  examine  the  cost  growth  experienced  by  the  13  projects  included  in  our 
reviews  for  the  past  3  years  that  were  confirmed  prior  to  fiscal  year  2009, 22 
we  measured  cost  growth  from  the  baseline  established  at  the  project’s 
confirmation  review.  In  some  instances,  this  baseline  was  established  before 
the  project  baseline  that  was  reported  to  Congress.  In  addition,  9  of  these 
projects  experienced  significant  cost  growth  in  excess  of  15  percent,  the 
point  at  which  NASA  is  required  to  notify  the  Congress  that  a  project  has 
exceeded  the  threshold  for  reporting.  See  table  3. 


Table  3:  Cost  Growth  from  Confirmation  for  Selected  Major  NASA  Projects  That  Established  Baselines  Prior  to  Fiscal  Year 
2009  (dollars  in  millions) 


Development  cost 


Project 

Baseline 

Current 

Difference 

Change  (%) 

Aquarius 

$193.0 

$227.3 

$34.3 

17.8 

Dawn 

$198.0 

$266.4 

$68.4 

34.5 

GLAST 

$384.0 

$418.8 

$34.8 

9.1 

Glory 

$159.0 

$337.6 

$178.6 

112.3 

Herschel 

$95.0 

$126.7 

$31.7 

33.4 

Kepler 

$313.0 

$388.7 

$75.7 

24.2 

LRO 

$421.0 

$451.3 

$30.3 

7.2 

MSL 

$969.0 

$1,802.0 

$833.0 

86.0 

NPP 

$513.0 

$780.1 

$267.1 

52.1 

OCO 

$187.0 

$230.2 

$43.2 

23.1 

SDO 

$597.0 

$667.0 

$70.0 

11.7 

SOFIA 

$306.0 

$1,128.4 

$822.4 

268.8 

WISE 

$192.0 

$191.8 

-$0.2 

-0.1 

Average 

$191.5 

54.99 

Total  development  cost 

$4,527.0 

$7,016.3 

$2,489.3 

Source:  GAO  analysis  of  NASA  data. 

N  ote:  "Baseline"  refers  to  the  cost  baseline  established  when  the  project  was  confirmed. 


If  the  changes  that  NASA  continues  to  implement  to  improve  its  acquisition 
management  have  their  intended  impact,  we  would  expect  to  see 
improvements  over  time  to  the  overall  performance  of  the  portfolio  of 


22  These  13  projects  include  5  projects  reviewed  this  year  and  8  projects  from  our  previous 
reports  in  this  series.  The  projects  are  of  analytical  interest  because  (1)  they  are  or  were  in 
the  implementation  phase  and  (2)  then  baselines  are  old  enough  to  begin  to  track  variances. 
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projects  in  maintaining  cost  and  schedule  baselines  established  at  their 
confirmation  reviews. 


Observations  from 
Our  Assessment  of 
Knowledge  Attained 
by  Key  Junctures 
in  the  Acquisition 
Process 


Many  of  NASA’s  projects  are  one-time  articles,  meaning  that  there  is  little 
opportunity  to  apply  knowledge  gained  to  the  production  of  a  second,  third, 
or  future  increments  of  spacecraft.  While  space  development  programs  are 
complex  and  difficult  by  nature  and  most  are  one-time  efforts,  NASA  is  still 
responsible  for  achieving  what  it  promises  when  requesting  and  receiving 
funds.  We  have  previously  reported  that  NASA  would  benefit  from  a  more 
disciplined,  knowledge-based  approach  to  its  acquisitions.  For  the  projects 
reviewed  this  year,  we  continue  to  identify  projects  that  have  not  met  best 
practice  standards  for  technology  maturity  and  design  stability  and  have 
experienced  challenges  in  development.  These  challenges  were  assessed 
based  on  knowledge  that,  according  to  acquisition  best  practices,  should  be 
attained  at  key  junctures  in  the  project  life  cycle  to  lessen  the  risks  to  the 
project. 


Technology  Challenges 

Projects  experiencing  technology 
challenges 

•  Ares  I 

•  Glory 

•  GPM 

•  GRAIL 

•  Juno 

•  JWST 

•  LADEE 

•  LDCM 

•  MMS 

•  MSL 

•  NPP 

•  Orion 

•  SOFIA 


During  the  course  of  our  review,  we  found  that  13  projects  had  experienced 
technology  issues,  such  as  a  lack  of  technology  maturity  for  both  critical 
and  heritage  technologies.  Specifically,  of  the  18  projects  that  had 
completed  the  preliminary  design  review — the  point  in  time  where  best 
practices  say  requisite  technology  maturity  should  be  reached  to  lessen 
risk — 11  projects  reported  moving  forward  with  immature  technologies.23 
Two  other  projects — MMS  and  NPP — reported  issues  with  immature 
technologies  for  instruments  that  were  being  developed  by  partners. 

Our  best  practices  work  has  shown  that  a  technology  readiness  level 
(TRL)  of  6 — demonstrating  a  technology  as  a  fully  integrated  prototype 
in  a  relevant  environment — is  the  level  of  maturity  needed  to  minimize 
risks  for  space  systems  entering  product  development.  For  NASA,  projects 
enter  development  following  the  project’s  preliminary  design  review  and 
confirmation  review.24  NASA’s  systems  engineering  policy  states  that  by  the 
preliminary  design  review  a  TRL  of  6  is  desirable  prior  to  integrating  a  new 
technology  in  a  project.26  Technology  maturity  is  a  fundamental  element 


23  The  Ares  I  and  Orion  projects  have  completed  their  preliminary  design  reviews,  but  have 
not  yet  held  confirmation  reviews. 

24  The  “product  development”  stage  in  GAO’s  knowledge-based  approach  is  equivalent  to 
“implementation”  in  NASA’s  life  cycle. 

25  NASA  Procedural  Requirements  (NPR)  7123. 1A,  NASA  Systems  Engineering  Processes 
and  Requirements,  Appendix  G,  paragraph  G.  19(b)  (Mar.  26,  2007). 
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of  a  sound  business  case,  and  its  absence  is  a  marker  for  subsequent 
problems,  especially  as  the  project  begins  more  detailed  design  efforts.26 

Similarly,  our  work  has  shown  that  the  use  of  heritage  technology — 
proven  components  that  are  being  modified  to  meet  new  requirements — 
can  also  cause  problems  when  the  items  are  not  sufficiently  matured  to 
meet  form,  fit,  and  function  standards  of  the  project  that  will  be  using  it 
by  the  preliminary  design  review.27  NASA  frequently  employs  heritage 
technologies  that  have  to  be  modified  from  their  original  form,  fit,  and 
function.  NASA’s  Systems  Engineering  Handbook  states  that  particular 
attention  must  be  given  to  heritage  systems  because  they  are  often  used  in 
architectures  and  environments  different  from  those  in  which  they  were 
designed  to  operate.  Further,  the  Handbook  states  that  modification  of 
heritage  systems  is  a  frequently  overlooked  area  in  technology  development 
and  that  there  is  a  tendency  by  project  management  to  overestimate 
the  maturity  and  applicability  of  heritage  technology  to  a  new  project. 

Our  work  has  shown,  and  NASA’s  own  guidance  concurs,  that  this  is  an 
area  that  is  frequently  underestimated  when  developing  project  cost 
estimates.  Although  NASA  distinguishes  critical  technologies  from  heritage 
technologies,  our  best  practices  work  has  found  critical  technologies  to 
be  those  that  are  required  for  the  project  to  successfully  meet  customer 
requirements,  regardless  of  whether  or  not  they  are  based  on  existing 
or  heritage  technology.  Therefore,  whether  technologies  are  labeled  as 
“critical”  or  “heritage,”  if  they  are  important  to  the  development  of  the 
spacecraft  or  instrument — enabling  it  to  move  forward  in  the  development 
process — they  should  be  matured  by  the  preliminary  design  review. 

NASA  is  making  progress  with  regard  to  adhering  to  best  practices 
standards  for  technology  maturity  at  the  preliminary  design  review  as 
the  number  of  projects  not  meeting  this  criteria  has  decreased  in  recent 
years.  Nearly  two-thirds  of  the  projects  in  our  current  review,  however, 
do  not  meet  this  standard.  See  figure  3  for  an  analysis  of  projects  that  we 
reviewed  in  the  past  3  years  that  held  their  preliminary  design  review  and 
the  percentage  of  those  projects  that  moved  into  implementation  with 
immature  technologies. 


26  Appendix  III  provides  a  description  of  the  metrics  used  to  assess  technology  maturity. 

27  Projects  will  modify  the  form,  tit,  and  function  of  a  heritage  technology  to  adapt  to  the 
new  environment.  For  example,  the  size  or  the  weight  of  the  component  may  change  or  the 
technology  may  function  differently  than  its  use  on  a  previous  mission. 
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Figure  3:  Percentage  of  Major  NASA  Projects  That  Moved  into  Implementation  with 
Immature  Technologies  at  the  Preliminary  Design  Review 
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Source:  GAO  analysis  of  NASA  data. 


Note:  Totals  may  not  add  to  100  percent  due  to  rounding. 

Proceeding  into  implementation  with  immature  technologies  increases 
a  project’s  risk  of  cost  and  schedule  overruns.  For  instance,  the  MSL 
project  was  given  approval  to  move  into  the  implementation  phase  despite 
reporting  that  seven  of  its  critical  technologies  were  not  mature  at  the 
time  of  its  preliminary  design  review.  At  the  critical  design  review  a  year 
later,  three  of  the  seven  critical  technologies  had  been  replaced  by  backup 
technologies  with  two  of  the  seven  were  still  assessed  as  immature, 
including  one  of  the  replacement  technologies.  Challenges  in  development 
contributed  to  the  MSL  project’s  26-month  schedule  delay  and  $834  million 
increase  in  total  life-cycle  costs.  In  another  example,  one  of  Glory’s 
main  instruments — the  Aerosol  Polarimetry  Sensor — was  assessed  as  an 
immature  critical  technology  at  the  project’s  preliminary  design  review, 
yet  the  project  was  approved  to  proceed  into  implementation.  Since  then, 
the  project  has  experienced  numerous  issues  with  development  of  that 
instrument,  resulting  in  over  a  year  delay  in  its  delivery  and  a  cost  increase 
to  the  project  of  over  $100  million. 


Page  17 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


Other  projects  in  formulation  are  allocating  extra  time  and  funding  in 
order  to  mature  critical  technologies  by  their  preliminary  design  review. 
By  investing  in  technology  development  early  in  the  project,  the  project 
may  safeguard  against  some  cost  and  schedule  growth  once  it  is  in  the 
implementation  phase.  For  example,  two  projects  in  the  formulation 
phase — ICESat-2  and  Solar  Probe  Plus — have  both  allocated  increased 
time  and  funding  for  development  of  their  multi-beam  laser  and  sunshield 
technologies,  respectively,  which  should  help  to  lessen  risk  to  the  projects 
moving  forward. 


Finally,  when  analyzing  the  number  of  reported  critical  technology 
development  efforts  by  the  projects  in  our  review,  we  found  four  of 
the  21  projects  in  our  review  reported  no  development  of  new  critical 
technologies,  while  another  8  projects  reported  development  of  only  one 
critical  technology.  Upon  presenting  this  data  to  senior  NASA  officials,  we 
were  told  that  it  appears  the  projects  did  not  accurately  identify  the  number 
of  critical  technologies  they  plan  to  develop  and  suggested  that  the  projects 
were  only  including  technologies  at  the  system  level.  We  plan  to  continue  to 
work  with  NASA  to  ensure  projects  are  accurately  identifying  their  critical 
technologies  to  assist  NASA  decision  makers  in  assessing  the  readiness  of 
projects  to  move  forward  in  their  development  life  cycles. 


Ten  of  the  12  projects  we  reviewed  that  had  held  their  critical  design 
review28  — the  point  in  time  where  best  practices  say  requisite  design 
maturity  should  be  reached  to  lessen  risk — did  not  meet  the  best  practices 
criteria  of  having  90  percent  engineering  drawings  releasable.  See  figure  4. 


Juno 

JWST 

MAVEN 

MMS 

MSL 

NPP 

SOFIA 


Design  Challenges 

Projects  experiencing  design  challenges 

•  Aquarius 

•  Glory 

•  GPM 


28  We  were  unable  to  determine  design  stability  for  the  SOFIA  project  as  some  data  was  not 
provided  to  us  for  review  by  NASA.  According  to  project  officials,  the  project  documentation 
did  not  transfer  in  its  entirety  from  Ames  Research  Center  to  Dryden  Flight  Research  Center. 
In  addition,  we  were  unable  to  determine  design  stability  for  the  MMS  project  as  it  did  not 
provide  us  with  detailed  drawing  count  data. 
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Figure  4:  Percentage  of  Engineering  Drawings  Releasable  at  CDR  for  Selected  NASA  Projects 

Percent  releasable 
100 


Projects  that  completed  CDR 


| _ |  Engineering  drawings  releasable  at  CDR 

Best  practices  criteria 
Source:  GAO  analysis  of  data  provided  by  NASA. 


We  have  previously  reported  that  NASA’s  acquisition  policy  does  not 
specify  a  metric  by  which  a  project’s  design  stability  is  measured  at 
the  critical  design  review.29  Guidance  in  NASA’s  Systems  Engineering 
Handbook,  however,  mirrors  the  best  practices  metric  that  at  least  90 
percent  of  engineering  drawings  should  be  releasable  by  the  critical  design 
review.  Discussions  with  project  officials  showed  the  metric  was  used 
inconsistently  to  gauge  design  stability.  For  example,  Goddard  Space  Flight 
Center  requires  greater  than  80  percent  drawings  released  at  the  critical 
design  review,  yet  several  project  officials  reported  that  the  “rule  of  thumb” 
for  NASA  projects  is  between  70  and  90  percent.  As  shown  in  figure  6 
above,  7  of  the  12  projects  reported  releasable  engineering  drawings  of 
less  than  70  percent,  lower  than  even  the  “rule  of  thumb”  used  by  several 
project  managers.  The  12  projects  averaged  having  only  62  percent  of 
their  engineering  drawings  releasable  at  their  critical  design  reviews, 
an  increase  from  the  less  than  40  percent  we  reported  last  year.  While 
the  average  has  improved,  it  is  still  well  below  the  best  practices  metric. 
Further,  nearly  all  of  the  projects  we  reviewed  over  the  last  three  years 
held  their  critical  design  review  without  90  percent  of  engineering  drawings 


29  GAO-06-218  and  GAO,  NASA:  Issues  Implementing  the  NASA  Authorization  Act  of  2010, 
GAO-11-216T  (Washington,  D.C.:  Dec.  1,  2010). 
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being  releasable — failing  to  meet  NASA  Systems  Engineering  Handbook 
guidance  and  our  best  practices  criteria  for  design  stability. 

Achieving  design  stability  allows  projects  to  “freeze”  the  design  and 
minimize  changes  in  the  future.  An  unstable  design,  on  the  other  hand,  can 
result  in  costly  re-engineering  and  re-work  efforts,  design  changes,  and 
schedule  slippage.  The  majority  of  the  12  projects  that  held  their  critical 
design  review  had  increases,  in  two  cases  well  over  100  percent,  to  the 
number  of  engineering  drawings  released  after  its  critical  design  review. 
According  to  NASA’s  systems  engineering  policy,  the  critical  design  review 
is  when  a  project’s  design  is  to  be  stable  enough  to  support  full-scale 
fabrication,  assembly,  integration  and  test.30  This  is  particularly  evident 
in  projects  in  our  review  that  held  their  critical  design  reviews  prior  to 
fiscal  year  2009,  or  projects  that  have  more  of  a  history  to  track  variances. 
As  shown  in  figure  5  below,  these  four  projects,  on  average,  had  a  107 
percent  increase  in  expected  engineering  drawings  after  the  critical  design 
review  after  having  only  36  percent  of  drawings  releasable  at  that  review. 
The  remaining  eight  projects  have  only  recently  held  their  critical  design 
review  in  fiscal  year  2009  or  later  and  have  not  reported  a  large  increase  in 
expected  drawings. 


30  NPR  7123. 1A,  Appendix  G,  paragraph  G.8  (Mar.  26,  2007). 


Page  20 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


Figure  5:  Comparison  of  Design  Drawing  Increase  for  Projects  with  CDR  prior  to  and 
since  Fiscal  Year  2009 
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Average  increase  in  expected  drawings  after  CDR 
Source:  GAO  analysis  of  data  provided  by  NASA. 

Some  of  the  projects  we  reviewed  in  the  past  3  years  pointed  to  other 
activities  that  occurred  prior  to  the  critical  design  review  as  evidence  of 
design  stability.  In  addition  to  releasable  engineering  drawings,  NASA 
often  relies  on  subject  matter  experts  in  the  design  review  process  and 
other  methods  to  assess  design  stability.  For  example,  the  Standing 
Review  Board31  provides  an  expert  assessment  of  the  technical  and 
programmatic  approach,  risk  posture,  and  progress  against  the  project 
baseline  at  key  decision  points  to  be  assured  that  the  project  has  a  stable 
design.  Furthermore,  some  projects  reported  using  engineering  models 
and  engineering  test  units  to  assess  design  stability.  For  example,  an  MMS 
project  official  reported  that  the  number  of  complete  engineering  test  units 
is  as  important,  if  not  more  so,  than  design  drawings.  By  using  engineering 
models  that  are  as  flight  ready  as  possible,  MMS  project  officials  reported 


31  For  KDP/milestone  reviews,  external  independent  reviewers  known  as  Standing  Review 
Board  (SRB)  members  evaluate  the  program/project  and,  in  the  end,  report  their  findings 
to  the  decision  authority.  For  a  program  or  project  to  prepare  for  the  SRB,  the  technical 
team  must  conduct  its  own  internal  peer  review  process.  This  process  typically  includes 
both  informal  and  formal  peer  reviews  at  the  subsystem  and  system  level.  NASA  Systems 
Engineering  Handbook,  paragraph  6.7.2. 1  (Dec.  2007). 
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that  they  can  see  where  problems  are  and  better  identify  risks.  In  addition,  a 
GPM  project  official  said  that  the  lack  of  releasable  drawings  at  the  critical 
design  review  did  not  have  a  serious  impact  in  terms  of  design  stability  as 
testing  was  almost  complete  on  the  engineering  test  units  and  flight  units 
were  already  designed  and  ready  to  begin  manufacturing.  The  Juno  project 
released  only  39  percent  of  engineering  drawings  at  its  critical  design 
review  and  project  officials  reported  that  they  used  engineering  models  for 
all  instruments  to  demonstrate  design  maturity  at  CDR  rather  then  released 
engineering  drawings.  The  Juno  project,  however,  experienced  a  46  percent 
increase  in  expected  number  of  engineering  drawings  after  its  CDR, 
indicating  that  the  design  was  not  stable. 


As  mentioned  above,  NASA  does  not  use  a  common  measure  to  assess 
design  stability  before  allowing  programs  to  move  from  the  design  phase 
to  the  test  and  integration  phases  of  the  development  process.  Our  studies 
and  others  have  found  that  significant  cost  growth  occurs  in  these  phases 
and,  in  some  instances,  has  tied  these  problems  to  issues  related  to  design. 
Moreover,  a  recent  study  by  the  National  Research  Council32  found  that 
the  critical  design  review  milestone  for  many  NASA  missions  may  be  held 
prematurely — driven  by  schedule  rather  than  driven  by  design  maturity. 
Regardless  of  how  stability  is  measured,  common  quantitative  measures 
employed  at  critical  design  review,  such  as  percentage  of  engineering 
drawings  that  are  in  a  releasable  state,  can  provide  evidence  that  the  design 
is  stable  and  provide  assurance  that  it  is  mature  and  will  meet  performance 
requirements.  These  measures  can  also  be  an  indication  to  decision  makers 
that  the  requisite  knowledge  has  been  attained  to  allow  the  project  to 
proceed  in  its  development  lifecycle  and  better  enable  them  to  assess  the 
performance  of  individual  projects  against  the  overall  portfolio  of  projects. 


Observations  on 
Other  Challenges  That 
Can  Affect  Project 
Outcomes 


In  addition  to  collecting  and  analyzing  data  on  the  attainment  of  knowledge 
at  key  junctures,  we  collected  and  assessed  data  on  five  additional  areas 
that  can  present  challenges  to  obtaining  positive  project  outcomes, 
including:  funding,  launch  vehicle,  development  partners,  parts,  and 
contractor  management.  Challenges  with  contractors  did  not  present  as  big 
a  challenge  to  projects  covered  by  this  review  compared  to  previous 
reports,  but  continues  to  warrant  monitoring  by  the  projects  and  other 
decision  makers  as  a  common  area  that  can  impact  project  execution.  The 
degree  to  which  each  area  challenged  project  execution  varied,  and  in  most 


32  The  National  Academies,  National  Research  Council,  Controlling  Cost  Growth  of  NASA 
Earth  and  Space  Science  Missions  (Washington,  D.C.  2010). 


Page  22 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


instances,  we  did  not  designate  any  specific  challenge  as  a  primary  factor 
for  cost  and/or  schedule  growth. 


Matching  funding  to  requirements  is  critical  to  the  success  of  complex 
acquisitions,  yet  it  is  often  insufficient  in  government  acquisitions.  Agencies 
tend  to  start  more  projects  than  can  be  afforded  and  often  have  to  make 
cuts  in  budgets  after  programs  begin  in  order  to  address  cost  increases  in 
highly  problematic  efforts.  Several  studies  have  highlighted  this  issue  with 
NASA  and  NASA’s  administrator  recently  stressed  the  need  to  ensure  that 
projects  are  affordable  before  they  are  started.  This  year,  we  identified  three 
projects  that  faced  significant  cost  and  schedule  problems  because  their 
original  funding  did  not  align  with  program  plans.  These  include  JWST,  Ares 
I,  and  Orion  and  they  represent  some  of  NASA’s  largest  investments.  In 
addition,  we  identified  10  projects  that  received  unanticipated  funding  from 
the  American  Recovery  and  Reinvestment  Act  of  2009.33  This  event  was  an 
anomaly,  and  according  to  NASA  officials,  it  carried  with  it  restrictions  and 
requirements  that  narrowed  the  scope  of  projects  it  could  be  applied  to  and 
required  additional  administrative  work,  which  initially  dissuaded  some 
projects  and  contractors  from  accepting  the  funds.  Nevertheless,  the 
stimulus  funding  enabled  NASA  to  mitigate  the  impact  of  cost  increases 
being  experienced  in  its  largest  projects  and  to  also  address  problems  being 
experienced  in  other  projects.  In  several  cases,  NASA  took  advantage  of  the 
funding  to  build  additional  knowledge  about  technology  or  design  before 
key  milestones. 

According  to  NASA  officials  and  independent  reviews,  the  projected 
budgets  for  JWST,  Ares  I,  and  Orion  were  inadequate  to  perform  work 
in  certain  fiscal  years.  In  November  2010,  an  independent  review  panel 
concluded  the  JWST  budget  baseline  accepted  at  the  confirmation  review 
did  not  reflect  the  most  probable  cost  with  adequate  reserves  in  each  year 
of  project  execution.  This  resulted  in  a  project  that  was  not  executable 
within  the  budgeted  resources.  According  to  the  review,  the  project  was 
able  to  stay  within  its  yearly  budget  allocation  by  deferring  planned  work 
in  the  budget  year  to  future  years.  This  approach  was  an  ineffective  control 
measure  as  costs  were  postponed  and  funded  from  a  subsequent  year’s 
allocation  at  a  cost  that  was  typically  two-  to  three-times  higher  because 
of  the  impact  of  the  deferrals  on  other  work.  Further,  the  panel  estimated 
that  the  project  will  need  an  additional  $1.4  billion  or  more  for  an  earliest 
launch  date  of  September  2015 — $500  million  of  which  will  be  needed  in 
fiscal  years  2011  and  2012.  Also,  as  we  have  reported  previously,  NASA 
initiated  the  Constellation  program  relying  on  the  accumulation  of  a 


Funding  Challenges 

Projects  experiencing  funding  challenges 

•  Aquarius 

•  Ares  I 

•  Glory 

•  GPM 

•  JWST 

•  Orion 

•  SOFIA 

Projects  that  received  ARRA  funding 

•  Aquarius 

•  Ares  I 

•  Glory 

•  GPM 

•  ICESat-2 

•  JWST 

•  LDCM 

•  OCO-2 

•  Orion 

•  SMAP 


Pub.  L.  No.  111-5. 
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large  rolling  budget  reserve  in  fiscal  years  2006  and  2007  to  fund  program 
activities  in  fiscal  years  2008  through  20 10.34  This  poorly  phased  funding 
plan  diminished  both  the  Ares  I  and  the  Orion  projects’  ability  to  deal 
with  technical  problems  and  funding  shortfalls  in  2010,  and,  in  part,  led 
the  President  to  propose  cancellation  of  the  program  in  the  fiscal  year 
2011  budget  submission.  An  independent  review  commissioned  by  the 
Administration  also  found  that  the  Ares  I  and  the  Orion  programs  did  not 
have  budget  profiles  that  matched  the  work  that  needed  to  be  done. 

With  regard  to  the  American  Recovery  and  Reinvestment  Act  of  2009 
(ARRA),  10  projects  used  these  additional  funds  to  offset  existing  funding 
issues,  such  as  covering  the  cost  of  delays  or  averting  “stop  work”  orders  to 
contractors,  or  to  lessen  risk  by  initiating  or  further  enhancing  technology 
development  efforts  and  long-lead  procurements  that  otherwise  would  not 
have  funded  at  that  time.  The  Science  Mission  Directorate  conducted  an 
extensive  analysis  on  how  best  to  utilize  the  funding,  because  officials  told 
us  that  these  additional  funds  would  not  necessarily  alleviate  all  technology 
development  or  other  schedule  delays,  and  in  some  cases  the  funds  would 
have  no  impact.  See  table  4  below  for  the  NASA  projects  in  our  review  that 
received  this  funding  and  how  these  funds  were  used. 


34  GAO,  NASA:  Constellation  Program  Cost  and  Schedule  Will  Remain  Uncertain  Until  a 
Sound  Business  Case  is  Established,  GAO-09-844  (Washington,  D.C.:  Aug.  26,  2009). 
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Table  4:  ARRA  Funding  for  Reviewed  NASA  Projects 

Project 

ARRA  funds 
(in  millions) 

Use  of  Funds 

Ares  1 

$102.4 

To  manufacture  and  assemble  engine  components  for  development  testing,  completion  of  a 
test  stand,  and  preparation  for  test  operations. 

Aquarius 

$8.6 

To  maintain  the  current  workforce  through  the  planned  launch. 

Glory 

$16.0 

To  maintain  the  current  workforce  through  the  planned  launch. 

GPM 

$32.0 

To  accelerate  construction  of  the  GPM  Microwave  Imager  (GMI)  instrument  to  ensure  the 
core  spacecraft  is  successfully  launched  at  the  earliest  possible  opportunity. 

ICESat-2 

$20.4 

To  mature  the  micro-pulse  laser  designs. 

JWST 

$75.0 

To  maintain  workforce  levels  and  achieve  the  earliest  possible  launch  date. 

LDCM 

$63.4 

To  initiate  development  of  the  thermal  infra-red  sensor  (TIRS). 

Other  LDCM  development. 

OCO-2 

$18.0 

To  acquire  long  lead  components  for  the  spacecraft  and  facilitate  instrument  development  in 
order  to  accelerate  and  enable  the  earliest  possible  OCO-2  launch. 

Orion 

$165.9 

To  avoid  workforce  reductions  and  mitigate  technical  challenges  with  its  launch  abort  system, 
landing  parachutes,  solar  arrays,  heatshield,  and  propulsion  systems. 

SMAP 

$64.0 

To  procure  long  lead  components  and  conduct  component  level  preliminary  design  reviews 
in  order  to  accelerate  the  launch  date. 

Source:  GAO  presentation  of  data  provided  by  NASA. 

Eight  of  21  projects  in  our  review  have  experienced  challenges  with  launch 
vehicles.  The  primary  concern  is  the  retirement  of  the  Delta  II  medium 
launch  vehicle.  Over  the  past  decade,  NASA  has  launched  about  60  percent 
of  its  science  missions  on  the  Delta  II.  NASA  plans  to  continue  to  use  the 
Delta  II  as  a  launch  vehicle  for  three  remaining  science  missions — Aquarius, 
GRAIL,  and  NPP — the  last  of  which  is  currently  scheduled  to  launch  in 
October  2011.  These  projects  have  identified  risks  associated  with  the  last 
flights,  such  as  the  availability  of  workforce  and  spare  parts  that  the 
projects,  along  with  NASA’s  Launch  Services  Program,  have  taken  steps  to 
mitigate. 

Our  recent  work  on  NASA’s  transition  plans  for  future  medium  launch 
vehicles  indicates  that  emerging  NASA  science  missions  will  face  increased 
risks  until  new  vehicles  are  certified.36  NASA  science  missions  requiring  a 
medium  class  launch  vehicle  that  are  approaching  their  preliminary  design 
review  face  uncertainties  committing  to  as-yet  uncertified  and  unproven 
launch  vehicles  that  will  eventually  replace  the  Delta  II.  Several  missions, 
including  the  SMAP  and  ICESat-2  missions  are  approaching  the  point  in  the 


Launch  Vehicle  Challenges 

Projects  experiencing  launch  vehicle 
challenges 

•  Glory 

•  GRAIL 

•  ICESat-2 

•  LADEE 

•  MAVEN 

•  NPP 

•  SMAP 

•  SPP 


35  GAO,  NASA:  Medium  Launch  Transition  Strategy  Leverages  Ongoing  Investments  but  Is 
Not  Without  Risk,  GAO-11-107  (Washington,  D.C.:  Nov.  22,  2010). 
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development  lifecycle  where  it  is  optimal  to  finalize  a  decision  on  launch 
vehicle.  NASA  plans  to  fill  the  gap  left  by  the  retirement  of  the  Delta  II  by 
eventually  certifying  the  Falcon  9  and  Taurus  II  vehicles36  for  use  by  NASA 
science  missions  in  the  relative  cost  and  performance  range  of  the  Delta 
II.  This  approach,  however,  is  not  without  risk  as  these  vehicles  are  largely 
unproven.  In  a  recent  report,  we  recommended  that  NASA  perform  detailed 
cost  estimates  to  determine  the  likely  costs  of  certification  of  these  new 
vehicles  and  provide  adequate  budgeting  for  the  risks  associated  with 
this  approach.37  NASA  concurred  with  this  recommendation  and  agreed 
to  provide  cost  estimates  for  certification  and  the  resolution  of  technical 
issues  during  certification  of  the  Falcon  9. 

Other  launch  challenges  beyond  the  Delta  II  transition  affected  projects  in 
our  review  this  year.  For  example,  the  Taurus  XL,  which  failed  during  the 
launch  of  OCO,  was  scheduled  to  return  to  flight  in  late  2010  for  the  Glory 
mission.  NASA  and  the  Taurus  XL  launch  vehicle  contractor  were  operating 
under  constrained  timelines  to  complete  Taurus  XL  return  to  flight 
activities;  however,  the  Glory  project  experienced  technical  challenges  that 
led  the  project  to  delay  the  launch  from  November  2010  to  February  2011, 
providing  enough  time  to  address  return  to  flight  activities.  A  malfunction  in 
the  ground  support  equipment  associated  with  the  Taurus  XL  launch  vehicle 
has  subsequently  delayed  launch  of  the  Glory  project  until  March  2011. 


Six  projects  reported  challenges  with  international  or  domestic 
development  partners  not  meeting  project  commitments  within  planned 
resources.  Project  officials  reported  several  reasons  why  development 
partners  were  unable  to  fulfill  their  obligations,  including  a  lack  of 
experience  in  producing  spacecraft  and  the  lack  of  adequate  funding.  For 
example,  delays  in  the  development  of  the  spacecraft  bus  by  Argentina’s 
National  Committee  of  Space  Activities  was  identified  as  the  reason  for  the 
Aquarius  project’s  15  percent  development  cost  increase  and  18-month 
schedule  slip  that  NASA  reported  to  the  Congress  in  February  2010.  Since 
that  time,  the  project  has  determined  that  the  launch  will  be  delayed  by  at 
least  another  5  months  for  a  total  delay  of  23  months.  Project  officials  said 
that  while  Argentina’s  National  Committee  of  Space  Activities  is  technically 
competent,  it  lacks  experience  in  managing  spacecraft  production  projects. 
Aquarius  project  officials  estimate  the  cost  impact  of  these  delays  to  be 
approximately  $35  million.  In  addition,  projects  also  experienced  challenges 


Development  Partner 
Challenges 

Projects  experiencing  development 
partner  challenges 

•  Aquarius 

•  GPM 

•  Juno 

•  LDCM 

•  MMS 

•  NPP 


36  NASA  provides  funding  to  SpaceX  and  Orbital  to  help  offset  International  Space  Station- 
related  development  costs  of  the  Falcon  9  and  the  Taurus  II,  respectively.  The  Falcon  9  and 
Taurus  II  are  intended  to  be  medium  class  launch  vehicles. 

37  GAO-11-107. 
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related  to  development  partners’  providing  adequate  funding  for  their 
contributions.  For  example,  the  GPM  project  identified  a  project  risk  that 
their  international  development  partner,  the  Japanese  Space  Agency,  may 
be  unable  to  fund  needed  launch  support  services  as  originally  planned. 

In  the  past  3  years,  we  reviewed  13  projects  that  established  their  baseline 
prior  to  fiscal  year  2009.  As  shown  in  table  5,  the  average  schedule  delay 
from  their  baselines  is  17.6  months  for  the  projects  with  foreign  or 
domestic  development  partners,  but  10.6  months  for  projects  that  had  no 
development  partner. 


Table  5:  Schedule  Growth  for  Selected  NASA  Projects  with  and  without  Development  Partners  Baselined  before  2009 

Project  with  partners 

Baseline 

(FY) 

Launch  delay 
(months) 

Projects 

without  partners 

Baseline 

(FY) 

Launch  delay 
(months) 

Dawn 

2007 

0 

Kepler 

2007 

9 

GLAST 

2007 

9 

SDO 

2007 

18 

Herschel 

2007 

21 

Glory 

2008 

21 

LRO 

2008 

8 

OCO 

2008 

5 

NPP 

2007 

42 

WISE 

2008 

1 

SOFIA 

2007 

12 

Aquarius 

2008 

23 

MSL 

2008 

26 

Average 

17.6 

10.6 

Source:  GAO  analysis  of  NASA  data. 


Although  the  cost  and  schedule  growth  for  some  of  the  projects  that  have 
development  partners  can  be  attributed  to  other  challenges,  for  example 
technology  or  design  issues,  there  are  instances  where  the  performance 
of  the  development  partners  was  the  primary  factor  of  cost  and  schedule 
growth.  For  example,  the  Aquarius,  NPP,  and  Herschel  projects  all 
experienced  significant  delays  as  a  direct  result  of  issues  related  to  their 
development  partners. 
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Parts  Challenges 


Projects  experiencing  parts  challenges 

•  Glory 

•  Juno 

•  LADEE 

•  LDCM 

•  MSL 

•  OCO-2 

•  RBSP 

•  TDRS  Replenishment 


While  most  of  the  projects  in  our  assessment  reported  challenges  related  to 
parts  quality  or  availability,  eight  projects  this  year  experienced  an  impact 
to  their  cost  or  had  to  make  alterations  to  their  schedules  as  a  result  of  the 
challenges.  According  to  NASA  officials,  parts  problems  are  not  uncommon 
for  projects,  and  NASA’s  testing  process  is  designed  to  identify  parts  failures 
at  the  component,  subsystem,  and  system  level  before  they  lead  to  mission 
failure.  For  example,  a  parts  quality  problem  discovered  during  the  testing 
and  integration  of  the  Glory  project  resulted  in  an  additional  $61million  in 
cost  and  delayed  the  project  by  17  months.  The  project  had  to  replace  the 
printed  wiring  board  of  the  spacecraft’s  single  board  computer  due  to 
reliability  problems  with  the  original  board.  In  addition,  the  project  recently 
discovered  excessive  wear  of  the  slip  ring  assembly  in  the  solar  arrays, 
resulting  in  an  additional  3-month  launch  delay.  The  MSL  project 
experienced  a  part  failure  associated  with  the  transition  joints  in  the 
propulsion  system  that  caused  the  joints  to  fail  under  load.  Project  officials 
reported  that  this  issue  was  realized  after  the  project  finished  building  its 
propulsion  system,  causing  the  project  to  rebuild  the  system  and  adopt  a 
new  joint  design.  The  transition  to  the  new  design  required  rework  and 
retest  of  the  descent  and  cruise  stages. 


NASA  centers  work  together  and  communicate  potential  systemic  issues 
with  parts.  For  example,  parts  personnel  at  Goddard  Space  Flight  Center 
maintain  a  center-level  parts  database,  which  links  to  the  agency-wide 
Government  Industry  Data  Exchange  Program  alert  system.38  GAO  has  an 
on-going  assessment  of  parts  quality  across  the  government  space  sector 
and  will  be  reporting  in  spring  2011  on  actions  being  taken  by  NASA  and 
other  agencies  to  prevent  and  mitigate  such  problems. 


Contractor  Management 
Challenges 

Projects  experiencing  contractor 
management  challenges 

•  Glory 

•  Juno 

•  JWST 

•  Orion 

•  RBSP 

•  SOFIA 


Five  projects  in  implementation  and  one  project  in  formulation  reported 
experiencing  contractor  challenges,  including  not  completing  work  on  time, 
not  identifying  risks  for  the  project,  and  inadequate  oversight.  Contractor 
management  challenges  have  been  reported  for  a  greater  number  of 
projects  and  with  a  greater  impact  for  projects  in  past  reports.  Although  the 
impact  of  this  challenge  on  projects  we  reviewed  this  year  has  diminished, 
as  contractors  spend  about  85  percent  of  NASA’s  annual  budget,  their 
performance  is  critical  in  terms  of  achieving  the  success  of  many  NASA 
missions.  As  a  result,  we  continue  to  identify  this  area  as  a  common  project 
challenge  that  can  contribute  to  cost  and  schedule  growth. 


38  Government  Industry  Data  Exchange  Program  (GIDEP)  is  a  partnership  between 
Government  Agencies  and  Industry  to  share  scientific  and  technical  information  through 
an  on-line  web-enabled  database.  GIDEP  alerts  report  a  problem  with  parts,  components, 
materials,  specifications,  software,  manufacturing  processes,  or  test  equipment  that  can 
cause  a  functional  failure. 
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In  one  case,  RBSP  project  officials  are  expecting  the  delivery  of  the 

Magnetic  Electron  Ion  Spectrometer  instrument  to  be  delayed  due  to 
the  time  a  vendor  is  taking  in  providing  needed  flight  hardware  for  the 
instrument.  Consequently,  the  project  has  re-planned  the  schedule  to 
accommodate  the  late  delivery  and  integration  of  the  instrument.  This 
re-plan  maintains  the  launch  readiness  date  by  reordering  the  observatory 
integration  and  test  flow  and  changing  selected  subsystem  and  instrument 
delivery  dates. 

In  another  example,  an  independent  review  panel  found  that  the  JWST 
project  did  not  have  staff  resident  at  the  prime  contractor  facility  to  help 
avoid  surprises,  especially  since  the  contract  represented  approximately 
half  of  the  JWST  project’s  budget.  The  panel  said  that  this  is  a  normal 
practice  and  is  done  for  other  projects  at  Goddard  Space  Flight  Center. 
Further,  while  project  officials  told  us  that  the  project’s  prime  contractor 
and  one  of  the  subcontractors  came  forward  after  continuation  with  large 
cost  increases  that  the  contractor  had  not  previously  identified  as  risks, 
the  panel  found  that  these  risks  had  been  identified  and  that  the  project 
had  asked  the  prime  contractor  to  submit  them  in  a  formal  proposal  before 
they  could  be  recognized  as  risks.  GAO  has  ongoing  work  to  review  NASA’s 
contractor  surveillance  and  oversight  practices  and  will  issue  a  report  later 
in  2011. 

Observations  about 
NASA’s  Continued 
Efforts  to  Improve 

Its  Acquisition 
Management 

In  response  to  GAO’s  designation  of  NASA’s  acquisition  management  as  a 
high  risk  area,39  NASA  developed  a  corrective  action  plan  to  improve  the 
effectiveness  of  its  program/project  management.40  The  plan  identifies  five 
areas  for  improvement — program/project  management,  cost  reporting 
process,  cost  estimating  and  analysis,  standard  business  processes,  and 
management  of  financial  management  systems — each  of  which  contains 
targets  and  goals  to  measure  improvement.  As  part  of  this  initiative,  the 
agency  is  continuing  its  implementation  of  a  new  cost  estimating  tool,  the 
Joint  Cost  and  Schedule  Confidence  Level,  to  help  project  officials  with 
management,  cost  and  schedule  estimating,  and  maintenance  of  adequate 
levels  of  reserves.  In  addition  to  the  corrective  action  plan,  NASA  is  in  the 
process  of  implementing  Earned  Value  Management  within  certain 
programs  and  specific  in-house  efforts  to  help  the  projects  monitor  the 
scheduled  work  done  by  its  contractors  and  employees;  however,  this 
management  tool  has  not  yet  been  institutionalized  within  the  NASA 

Centers.  These  two  efforts,  in  addition  to  other  improvements  NASA  is 

39  GAO,  High-Risk  Se?~ies:  An  Update,  GAO-07-310  (Washington,  D.C.:  Jan.  2007). 

40  National  Aeronautics  and  Space  Administration,  Plan  for  Improvement  in  the  GAO  High- 
Risk  Area  of  Contract  Management  (Oct.  31,  2007). 
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making  to  address  acquisition  management,  are  positive  steps  toward 
addressing  NASA’s  issues  with  meeting  cost  and  schedule  baselines.  It  is, 
however,  too  early  to  assess  their  impact  on  NASA’s  performance. 
Additionally,  NASA’s  progress  could  be  hindered  by  the  continued  lack  of  a 
consistent  measure  for  ensuring  design  stability  as  well  as  little 
transparency  with  regard  to  costs  for  projects  in  the  early,  critical  phases  of 
development,  both  of  which  are  key  to  ensuring  that  internal  and  external 
decision  makers  are  well  informed.  We  recently  raised  both  issues  as 
potential  impediments  to  success  in  congressional  testimony  and  plan  to 
recommend  improvements  in  a  separate  report.41 

Joint  Cost  and  Schedule  NASA’s  Joint  Cost  and  Schedule  Confidence  Level  (JCL)  initiative,  adopted 

Confidence  Levels  Being  in  January  2009,  is  a  point-in-time  estimate  that  includes,  among  other 

Implemented  things,  all  cost  and  schedule  elements,  incorporates  and  quantifies  known 

risks,  assesses  the  impacts  of  cost  and  schedule  to  date,  and  addresses 
available  annual  resources.  The  primary  goals  of  the  JCL  are  to  help  project 
officials  with  management,  cost  and  schedule  estimating,  and  maintenance 
of  adequate  levels  of  reserves;  provide  assurance  to  stakeholders  that  NASA 
will  meet  cost  and  schedule  targets;  and  to  provide  transparency  on  the 
effects  of  funding  changes  on  the  probability  of  meeting  cost  and  schedule 
commitments.  NASA  requires  that  a  JCL  be  developed  prior  to  the 
confirmation  review.  NASA  policy  also  requires  that  projects  be  baselined 
and  budgeted  at  the  70  percent  confidence  level  and  funded  at  a  level 
equivalent  to  at  least  the  50  percent  confidence  level  for  the  project.42 
According  to  NASA  officials,  this  would  include  reserves  held  at  the 
directorate  and  project  level.  The  total  amount  of  reserves  held  at  the 
project  level  varies  based  on  where  the  project  is  in  its  life  cycle.  The 
reserves  represent  the  amount  of  estimated  costs  that  are  not  allocated  to 
the  specific  project  sub-elements.  See  figure  6  for  a  visual  depiction  of  this 
funding  allocation. 


41  GAO,  Additional  Cost  Transparency  and  Design  Criteria  Needed  for  National 
Aeronautics  and  Space  Administration  (NASA)  Projects,  GAO-11-346R  (Washington,  D.C.: 
Mar.  3,  2011). 

42  NASA  Policy  Directive  1000.5A,  Policy  for  NASA  Acquisitions,  paragraphs  1(h)(1)(a)  and 
1(h)(2)  (Jan.  15,  2009). 
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Figure  6:  Notional  Allocation  of  Reserves  under  the  70  Percent  Confidence  Level 
Funding  Requirements 

Probability  (confidence  level)  of  meeting  cost  target 


Source:  GAO  analysis  of  NASA  policy. 


Note:  The  amount  of  project  reserves  varies  as  the  project  moves  through  its  life  cycle. 


NASA’s  Associate  Administrator  for  the  Science  Mission  Directorate 
indicated  that  adoption  of  the  new  JCL  process  will  reduce  NASA’s  portfolio 
because  the  cost  estimating  will  be  more  accurate  at  the  70  percent 
confidence  level,  reflecting  higher  costs  from  the  outset  to  avoid  higher 
cost  overruns  in  the  future,  and  as  a  result,  NASA  will  have  fewer  dollars 
available  to  start  new  projects. 

Five  out  of  the  21  projects43  in  our  review  have  recently  completed  the 
JCL  process,  and  several  others  are  in  the  process  of  conducting  a  JCL 
analysis.  NASA  is  still  in  the  process  of  refining  the  tools  used  to  create 
the  JCL  based  on  feedback  from  the  projects.  As  NASA  evolves  its  cost 
estimation  processes  and  as  we  continue  to  conduct  our  reviews  of  the 
projects  that  have  gone  through  the  JCL  process,  we  can  better  assess  the 
impact  this  initiative  has  on  the  projects’  ability  to  meet  cost  and  schedule 
commitments  and  to  address  potential  cost  and  schedule  drivers. 


43  Seven  of  the  21  projects  were  not  required  to  complete  the  JCL  process  at  the  time  of  our 
review. 
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Implementation  of  Earned 
Value  Management  at 
NASA  Centers  in  Progress 


Earned  value  management  (EVM)  is  a  program  management  tool  that 
integrates  the  technical,  cost,  and  schedule  parameters  of  a  contract  and 
uses  those  parameters  to  measure  cost  and  schedule  variances.  During  our 
review,  we  found  that  implementation  of  earned  value  management  is 
occurring  within  11  projects  and  earned  value  data  is  reported  by  projects 
on  a  monthly  basis  to  upper  level  project  management.  While  earned  value 
management  is  being  used  by  these  projects,  it  is  not  yet  clear  it  is  being 
used  consistently  by  the  projects  as  a  tool  for  managing  cost  and  schedule. 
According  to  a  briefing  from  the  NASA  Advisory  Council’s  Audit,  Finance, 
and  Analysis  Committee,  NASA’s  goal  is  to  develop  and  deploy  an  agency¬ 
wide  EVM  capability  that  is  compliant  with  generally  accepted  standards.44 
At  this  time,  only  the  Jet  Propulsion  Laboratory,  a  federally  funded  research 
and  development  center  and  not  a  NASA  Center,  has  a  compliant  system. 


If  implemented  appropriately,  EVM  provides  objective  reports  of  project 
status,  produces  early  warning  signs  of  impending  schedule  delays  and 
cost  overruns,  and  can  identify  specific  development  efforts  contributing 
to  those  overruns.  For  example,  MSL’s  June  2010  EVM  report  identified  the 
avionics  and  actuators  as  the  primary  drivers  of  the  project’s  cost  overruns. 
In  particular,  the  data  showed  that  ongoing  unplanned  technical  issues  with 
three  of  the  heritage  avionics  technologies  would  likely  result  in  a  cost 
overrun  of  $11.5  million.  More  consistent  use  of  this  management  tool  could 
help  address  the  project  challenges  identified  earlier  in  this  report  that 
threaten  the  project’s  cost  and  schedule  during  project  development.  The 
EVM  data  we  received  from  NASA  was  not  received  in  a  timely  manner  and 
was  incomplete.  As  a  result,  we  were  unable  to  perform  a  detailed  analysis 
by  project  to  provide  our  own  determination  of  whether  the  information 
provided  by  the  contractors  is  accurate  and  could  be  relied  on  by  the 
projects  and  management  as  a  tool  to  assess  progress.  We  plan  to  conduct 
a  more  thorough  analysis  of  EVM  data  in  ongoing  work  and  in  future 
iterations  of  this  work. 


Transparency  and 
Accountability  Not 
Sufficient  to  Support 
Oversight 


NASA’s  initiatives  aimed  at  improving  cost  estimating  and  management 
oversight  are  positive  steps.  However,  we  recently  testified  that  NASA  does 
not  provide  enough  transparency  during  early  project  development  to 
provide  Congress  with  sufficient  information  to  conduct  oversight  and 
ensure  earlier  accountability.46  Currently,  NASA  does  not  share  cost  and 
schedule  information  for  projects  in  the  early,  critical  phases  of 
development  and  only  makes  this  information  public  after  the  projects  have 


44  American  National  Standards  Institute/Electronic  Industries  Alliance  Standard,  Earned 
Value  Management  Systems,  ANSI/EIA-748-B-2007  approved  July  9,  2007. 

46  GAO-11-216T. 
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been  formally  approved  to  enter  implementation.  Projects  establish 
preliminary  cost  estimates  in  the  formulation  phase.  These  estimates, 
however,  are  for  planning  purposes  only  as  they  enable  NASA  decision 
makers  to  better  manage  the  overall  portfolio  of  projects.  NASA  does  not 
report  deviations  from  these  preliminary  estimates  to  the  Congress.  In 
addition,  NASA  does  not  report  information  on  what  has  been  spent  to  date 
on  the  projects  in  formulation,  as  it  does  in  its  annual  budget  submission  for 
projects  in  implementation.  To  add  some  perspective  to  this  timing,  neither 
the  Ares  I  nor  Orion  projects  has  reached  the  implementation  phase,  despite 
having  spent  over  $9  billion  dollars  combined;  and  JWST  just  reached  this 
point  in  2008,  despite  having  spent  nearly  $2  billion  before  then. 

Despite  the  absence  of  publicly  reported  cost  and  schedule  baselines  to 
measure  the  progress  of  the  project,  cost  growth  and  schedule  delays  can 
and  do  occur  during  the  formulation  phase.  NASA’s  internal  analysis  of 
past  projects  indicates  that  there  is  an  average  of  14  percent  growth  in  the 
development  cost  estimates  during  the  formulation  phase.  While  there  is  a 
need  to  allow  projects  a  period  of  time  for  discovery  and  to  pursue  different 
concepts — particularly  highly  complex  efforts  such  as  JWST — inadequate 
transparency  into  their  progress  for  what  sometimes  amounts  to  5  or  more 
years  can  preclude  effective  oversight  and  accountability  and  make  it  even 
more  difficult  to  stop  projects  that  are  not  on  track  to  meet  the  agency’s 
goals  with  available  resources.  Additional  insight  to  costs  could  better 
enable  Congress  to  make  more  informed  decisions  when  approving  the 
projects  through  the  annual  appropriations  process. 

In  addition,  a  recently  released  report  from  the  Independent  Comprehensive 
Review  Panel46  concerning  problems  affecting  the  JWST  program 
concluded  that  significant  changes  are  still  needed  in  NASA’s  oversight 
and  accountability  functions  to  ensure  that  programs  base  their  decisions 
on  sound  knowledge.  The  panel  noted  that  NASA’s  governance  policy 
is  inconsistent  with  accountability  for  project  execution.  In  particular, 
the  panel  found  that  a  lack  of  clear  lines  of  authority  and  accountability 
contributed  to  a  lack  of  executive  leadership  in  resolving  the  broken  JWST 
life-cycle  cost  baseline.  Additionally,  the  study  found  that  JWST’s  flawed 
budget  should  have  been  discovered  as  part  of  the  Goddard  Spaceflight 
Center’s  execution  responsibility,  but  the  interpretation  of  the  agency’s 
governance  policy  on  the  role  of  the  center  in  this  regard  is  ambiguous  and 
not  interpreted  uniformly  within  NASA.  As  a  result,  the  report  noted  that 


46  Jet  Propulsion  Laboratory,  James  Webb  Space  Telescope  (JWST)  Independent 
Comprehensive  Review  Panel  (ICRP):  Final  Report,  JPL  D-67250  (Pasadena,  Calif.:  Oct.29, 
2010). 


Page  33 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


ongoing,  regular  independent  assessment  and  oversight  processes  at  the 
agency  are  missing. 

Project  Assessments 

The  2-page  assessments  of  the  projects  we  reviewed  provide  a  profile  of 
each  project  and  describe  the  challenges  we  identified  this  year,  as  well  as 
challenges  that  we  have  identified  in  the  past.  On  the  first  page,  the  project 
profile  presents  a  general  description  of  the  mission  objectives  for  each  of 
the  projects;  a  picture  of  the  spacecraft  or  aircraft;  a  schedule  timeline 
identifying  key  dates  for  the  project;  a  table  identifying  programmatic  and 
launch  information;  a  table  showing  the  current  baseline  year  cost  and 
schedule  estimates  and  the  February  2011  cost  and  schedule  data;  a  table 
showing  the  challenges  relevant  to  the  project;  and  a  project  summary 
narrative.  To  maintain  information  on  challenges  the  projects  experience 
over  their  lifetime,  we  continued  to  identify  project  challenges  that  were  as 
previously  reported.  On  the  second  page  of  the  assessment,  we  provide  an 
analysis  of  the  project  challenges  and  the  extent  to  which  each  project  faces 
cost,  schedule,  or  performance  risk  because  of  these  challenges.  NASA 
project  offices  were  provided  an  opportunity  to  review  drafts  of  the 
assessments  prior  to  their  inclusion  in  the  final  product,  and  the  projects 
provided  both  technical  corrections  and  more  general  comments.  We 
integrated  the  technical  corrections  as  appropriate  and  characterized  the 
general  comments  below  the  project  update.  See  figure  7  below  for  an 
illustration  of  the  layout  of  each  2-page  assessment. 
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Figure  7:  Illustration  of  Project  2-Page  Summary 
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well  an  current  schedule  dcluys  awuM-ialrd  with  the  SAi'll  iinlmmenls  tn-ing  pros  Mini  by  CONAK.  These 
M-lwtlule  sli|n  inen-used  NASA's  rosin  by  un  estimated  $:&5  million  in  the  punt.  IVoJnl  officials  slalnl  that 
tlw  |>riinao'  coal  driver  uMNoriatcri  with  llw  lutim-b  delay  In  sniffing  costs,  estlmulnl  to  Iw  approximately  $1.1* 
million  l-Tifllier,  the  prqjcci  fmtvnl  W.fi  millioti  under  the  American  Recovery  and  KriinemiiHiii  Act  of 
2lk)H  that  was  lined  to  nuintain  the  current  Aqporiua  workforce  throitgb  launch. 

Other  Issues  to  be  Monitored:  During  thermal  vacuum  testing  on  llw  spurrrrafl  Inis,  INVAP  (Uncovered 
a  proldem  with  the  s|»iwr»ft'*  pmpuhatm  systems  Ihruslris  lluii  has  eontrilniUsi  lo  delaying  Um-  Iuiiim-Ii 
until  June  2011.  After  an  analysis  of  the  Dual  TluiLsti-r  Module,  llw  Aquariin/SAC-D  learn  deti-miinrd  that 
tlw  iwoblctu  was  likely  due  to  one  or  mon-  pnnsshiral  Ihiim-s  in  the  li-st  process  at  the  manufacliuer  or  its 
vendor.  Itcfurirtvlinn-rit  of  all  of  the  Dual  Thninlcr  Moduli-  flight  units  is  complete,  ami  thi-  flight  units  were 
rv-lntegniti-d  with  the  olsiervatoty.  INVAP  plana  to  complete  Integration  ami  testing  hy  November  2010. 

Project  Office  Comment* 

The  Aquarius  |ttojixi  proviik-d  technical  rumim-nls  lo  a  draft  of  this  unm-mtnml,  w  liicll  w  ife  incogniransl 
as  ap|  nop  rule  Tlw  prqjcct  officials  also  coriunmted  dial  NASA  and  CONAK  will  continue  lo  work  together 
to  meet  the  earliest  puwiihie  lauiwh  tlale. 


© 


© 


O  General  description  of  mission’s  science  objectives 

0  Illustration  of  spacecraft  or  aircraft 

0  Schedule  timeline  identifying  key  dates  for  the  project  including  when 
the  project  began  formulation,  major  design  reviews,  confirmation  to 
begin  the  implementation  phase,  and  scheduled  launch  readiness 

0  Project  Essentials  Programmatic  information  including  the  responsible 
NASA  center,  international  or  domestic  partners,  major  contractors,  and 
launch  information 

©  Project  Performance  Cost  and  schedule  baseline  estimates  and  the 
latest  estimate  updates  as  of  February  201 1 


0  Project  Challenges  Summary  listing  the  challenges  facing  the  project 
based  on  a  successful  acquisition  business  case 

©  Project  Summary  Brief  narrative  describing  current  status  of  the 
project  with  regard  to  the  challenges  identified 

©  Project  Update  Analysis  of  project  challenges  and  the  extent  to  which 
each  project  faces  cost,  schedule,  or  performance  risk  because  of  these 
challenges 

O  Project  Office  comments  General  comments  provided  by  the 
cognizant  project  office 


Source:  GAO  analysis. 
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Common  Name:  Aquarius 


Aquarius 

Aquarius  is  a  satellite  mission  developed  by  NASA 
and  the  Space  Agency  of  Argentina  (Comision 
Nacional  de  Actividades  Espaciales,  CONAE)  to 
investigate  the  links  between  the  global  water  cycle, 
ocean  circulation,  and  the  climate.  It  will  measure 
global  sea  surface  salinity.  The  Aquarius  science 
goals  are  to  observe  and  model  the  processes  that 
relate  salinity  variations  to  climatic  changes  in  the 
global  cycling  of  water  and  to  understand  how  these 
variations  influence  the  general  ocean  circulation.  By 
measuring  salinity  globally  for  3  years,  Aquarius  will 
provide  a  new  view  of  the  ocean’s  role  in  climate. 


Source:  Aquarius  Project  Office  (artist  depiction). 
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\  Implementation 
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Formulation  Preliminary 
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Critical  design 

GAO 

Launch 

start  design  review 

confirmation 

review 
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readiness  date 

(12/03)  (6/05) 

(9/05) 

(9/06) 

(12/10) 

(6/11) 

Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory  (JPL)* 
International  Partner:  Argentina's  National  Committee  of 
Space  Activities  (CONAE) 

Major  Contractors:  In-house  development 

Projected  Launch  Date:  June  201 1 
Launch  Location:  Vandenberg  AFB,  Calif. 

Launch  Vehicle:  Delta  II 

Mission  Duration:  3  years  for  Aquarius  mission 

5  years  for  SAC-D  (CONAE)  mission 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2008) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$241 .8 

$279.0 

15.4% 

Formulation  Cost 

$35.5 

$35.6 

0.3% 

Development  Cost 

$192.7 

$227.3 

18.0% 

Operations  Cost 

$13.6 

$16.1 

18.4% 

Launch  Schedule 

7/2009 

6/201 1 

23  months 

'JPL  is  a  federally  funded  research  and  development  center 

Project  Summary 

The  launch  of  Aquarius  has  been  delayed  from  the  July 
2009  baseline  to  June  2011  because  of  delays  in  CONAE’s 
spacecraft  development  and  problems  with  the  propulsion 
system  thrusters.  The  launch  delay,  which  added  costs 
to  the  project,  prompted  NASA  to  report  to  the  Congress 
in  February  2010  that  the  Aquarius  project  exceeded  its 
development  cost  and  schedule  baselines  by  15  percent 
and  more  than  6  months,  respectively.  NASA  completed  its 
development  of  the  Aquarius  instrument,  which  is  currently 
being  integrated  with  the  Argentine-developed  spacecraft. 
Project  officials  estimated  the  cost  of  the  past  schedule  slips 
to  be  about  $35.5  million. 


Recent/Continuing  Project 
Challenges 

»-  Development  Partner  Issues 
»-  Funding  Issues 

Previously  Reported  Challenges 

»-  Design  Stability 
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Common  Name:  Aquarius 


Aquarius 

Project  Update 

NASA  reported  to  Congress  in  the  agency’s  fiscal  year  2011  budget  estimates  that  the  Aquarius  mission’s 
development  costs  had  grown  by  15  percent  from  its  2008  baseline.  Additionally,  the  project’s  current  June 
2011  launch  date  represents  a  23-month  schedule  slip.  These  cost  and  schedule  overruns  are  due  to  delays 
by  the  international  partner. 

Development  Partner  Issues:  According  to  project  officials  and  budget  documents,  delays  in  the 
development  of  the  spacecraft  bus  by  CONAE  were  responsible  for  the  15  percent  development  cost 
increase  and  18-month  schedule  slip  that  NASA  reported  to  Congress  in  February  2010.  Since  that  time, 
the  project  has  determined  that  the  launch  will  be  delayed  by  another  5  months  to  June  2011,  for  a  total 
delay  of  23  months.  To  facilitate  the  work  of  its  partners,  the  Jet  Propulsion  Laboratory  (JPL)  project  team 
said  that  it  appointed  a  chief  mission  engineer  to  help  facilitate  upcoming  tests  and  reviews;  however,  JPL 
officials  stated  that  they  have  not  had  full  access  to  INVAP,  CONAE’s  prime  contractor,  due  to  contractual 
agreements  between  INVAP  and  CONAE.  Additionally,  CONAE  was  responsible  for  flying  the  instrument  to 
Vandenberg  Air  Force  Base  for  launch  but  could  not  find  a  viable  commercial  aircraft.  Project  officials  said 
that  they  are  working  with  the  U.S.  Air  Force  to  secure  a  no-cost  flight  for  the  integrated  satellite,  but  may 
have  to  pay  for  the  flight  at  a  cost  of  approximately  $1  million. 

Funding  Issues:  Since  no  funds  are  being  exchanged  between  the  U.S.  and  Argentina  for  this  project, 

NASA  bears  the  costs  it  incurs  associated  with  any  schedule  delays.  Project  officials  told  us  that  all  of  the 
project’s  contingency  reserves  have  been  eroded  due  to  past  schedule  delays  with  the  spacecraft  bus  as 
well  as  current  schedule  delays  associated  with  the  SAC-D  instruments  being  provided  by  CONAE.  These 
schedule  slips  increased  NASA’s  costs  by  an  estimated  $35.5  million  in  the  past.  Project  officials  stated  that 
the  primary  cost  driver  associated  with  the  launch  delay  is  staffing  costs,  estimated  to  be  approximately  $4.9 
million.  Further,  the  project  received  $8.6  million  under  the  American  Recovery  and  Reinvestment  Act  of 
2009  that  was  used  to  maintain  the  current  Aquarius  workforce  through  launch. 

Other  Issues  to  be  Monitored:  During  thermal  vacuum  testing  on  the  spacecraft  bus,  INVAP  discovered 
a  problem  with  the  spacecraft’s  propulsion  systems  thrusters  that  has  contributed  to  delaying  the  launch 
until  June  2011.  After  an  analysis  of  the  Dual  Thruster  Module,  the  Aquarius/SAC-D  team  determined  that 
the  problem  was  likely  due  to  one  or  more  procedural  issues  in  the  test  process  at  the  manufacturer  or  its 
vendor.  Refurbishment  of  all  of  the  Dual  Thruster  Module  flight  units  is  complete  and  the  flight  units  were 
re-integrated  with  the  observatory.  INVAP  planned  to  complete  integration  and  testing  by  November  2010. 

Project  Office  Comments 

The  Aquarius  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  The  project  officials  also  commented  that  NASA  and  CONAE  will  continue  to  work  together 
to  meet  the  earliest  possible  launch  date. 
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Common  Name:  Ares  I 


Ares  I  Crew  Launch  Vehicle 


NASA’s  Ares  I  Crew  Launch  Vehicle  was  designed  to 
carry  the  Orion  Crew  Exploration  Vehicle  into  low 
Earth  orbit  for  missions  to  the  International  Space 
Station  and  the  Moon  as  part  of  the  Constellation 
Program.  The  mission  of  the  Ares  I  project  was  to 
deliver  a  safe,  reliable,  and  affordable  launch  system 
with  a  24.5-metric  ton  lift  capability. 


Formulation 

Implementation 

▲ 

▲ 

▲ 

▲ 

▲ 

Formulation 

Preliminary 

GAO 

Critical  design 

Launch 

start 

design  review 

review 

review 

readiness  date 

(09/05) 

(9/08) 

(12/1 0) 

(9/11) 

(3/15) 

Project  Essentials 

NASA  Center  Lead:  Marshall  Space  Flight  Center 
Partners:  None 

Major  Contractors:  AlliantTechsystems,  Pratt  and  Whitney 
Rocketdyne,  Boeing 


Project  Performance 
(then  year  dollars  in  millions) 

Latest 
(Feb.  2011) 

Preliminary  Estimate  of 

Project  Life  Cycle  Cost*  $1 7,000  to  $20,000 


Projected  Launch  Date:  March  2015 
Launch  Location:  Kennedy  Space  Center,  Fla. 
Launch  Vehicle:  Ares  I 

Mission  Duration:  N/A 


*  This  estimate  is  preliminary,  as  the  project  is  in  formulation 
and  there  is  still  uncertainty  in  the  value  as  design  options 
are  explored.  NASA  uses  these  estimates  for  planning 
purposes.  This  estimate  is  for  the  Ares  I  vehicle  only 


Launch  Schedule  3/2015 


Recent/Continuing  Project 
Challenges 

»-  Funding  Issues 
»-  Technology  Issues 


Project  Summary 

The  President’s  fiscal  year  2011  budget  proposed 
cancellation  of  the  Ares  I  project  leading  to  uncertainty, 
both  financial  and  programmatic,  within  the  project.  Given 
constrained  resources,  the  project  prioritized  work  and  did 
not  accomplish  some  of  the  work  originally  planned  for 
2010;  however,  it  successfully  tested  Development  Motor  2 
to  gain  data  on  project  elements.  In  early  fall  2010,  Congress 
passed  the  NASA  Authorization  Act  of  2010  directing  NASA 
to  develop  a  space  launch  system  and  crew  vehicle  for 
missions  utilizing  existing  Ares  I  contracts  and  capabilities 
to  the  extent  practicable. 
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Common  Name:  Ares  I 


Ares  I  Crew  Launch  Vehicle 


Project  Update 

The  President  proposed  cancellation  of  the  Constellation  Program,  including  the  Ares  I  project,  in  the 
fiscal  year  2011  budget  request.  This  proposal  led  to  much  debate  within  Congress  and  uncertainty,  both 
financial  and  programmatic,  within  the  project.  As  a  result,  the  project  prioritized  work  for  the  year  and  did 
not  complete  some  of  the  work  originally  planned  for  2010.  In  early  fall  2010,  Congress  passed  the  NASA 
Authorization  Act  of  2010,  which  directed  NASA  to  develop  a  space  launch  system  and  crew  vehicle  for 
missions  to  near  earth  orbit  and  regions  of  space  beyond  low-Earth  orbit  no  later  than  December  2016.  In 
developing  this  vehicle,  Congress  directed  the  agency  to  extend  or  modify  existing  vehicle  development  and 
associated  contracts  to  the  extent  practicable. 

Funding  Issues:  The  Ares  I  project  received  over  $102  million  under  the  American  Recovery  and 
Reinvestment  Act  of  2009  (ARRA)  that  was  used  to  manufacture  and  assemble  engine  components  for 
development  testing,  completion  of  a  test  stand,  and  preparation  for  test  operations.  However,  project 
officials  explained  that  due  to  a  series  of  budgetary  constraints  for  the  first  4  months  of  fiscal  year  2010  that 
roughly  offset  the  amount  gained  from  the  ARRA  funding,  the  project  could  not  perform  all  of  its  originally 
planned  work.  While  initially  parts  of  the  project  were  able  to  maintain  momentum,  termination  liability 
issues  identified  in  June  2010  caused  the  three  project  prime  contractors  to  stop  certain  portions  of  the 
work  on  their  respective  contracts.  At  this  time,  the  project  redirected  its  funding  to  activities  that  would 
potentially  benefit  NASA’s  goals  and  objectives  beyond  the  current  fiscal  year.  For  example,  in  August  2010, 
the  project  successfully  tested  Development  Motor  2  (DM-2).  The  DM-2  test  was  conducted  to  gain  data  on 
project  elements  tested  including  the  redesigned  rocket  nozzle,  new  insulation,  and  the  motor  casing’s  liner. 
According  to  project  officials,  the  project  was  flexible  in  its  planning  while  it  maintained  the  program  of 
record  during  fiscal  year  2010. 

Technology  Issues:  The  Ares  I  project  has  been  working  to  mitigate  several  challenges  related  to  the 
development  of  heritage  technology.  However,  given  the  funding  uncertainty  that  has  surrounded 
Ares  I,  the  project  has  been  unable  to  implement  the  mitigation  strategies.  For  example,  last  year,  NASA 
identified  thrust  oscillation  as  a  technical  issue.  Thrust  oscillation,  which  causes  shaking  during  launch 
and  ascent,  occurs  in  some  form  in  every  solid  rocket  engine.  Computer  modeling  indicated  that  there  was 
a  possibility  that  the  magnitude  and  frequency  of  thrust  oscillation  within  the  first  stage  would  be  outside 
the  limits  of  the  Ares  I  design  and  could  cause  excessive  vibration  in  the  Orion  capsule  and  threaten  crew 
safety.  According  to  project  officials,  the  project  plans  to  mitigate  the  risk  by  adding  damper  and  isolation 
techniques  at  the  interface  between  the  launch  vehicle  and  the  Service  Module.  However,  this  risk  cannot 
be  closed  until  funding  is  obtained  to  implement  the  mitigation  strategy.  Furthermore,  vibroacoustics — the 
pressure  of  the  acoustic  waves  produced  by  the  firing  of  the  Ares  I  first  stage  and  the  rocket’s  acceleration 
through  the  atmosphere — continues  to  be  a  concern  to  the  project.  Vibroacoustics  may  cause  unacceptable 
structural  vibrations  throughout  Ares  I  and  Orion  and  force  NASA  to  qualify  components  to  higher  vibration 
tolerance  thresholds  than  originally  expected.  According  to  the  project,  the  global  mitigation  strategy  for  the 
excessive  vibration  has  been  on  hold  due  to  budget  constraints.  The  project  is  unable  to  finalize  the  design 
without  knowing  the  final  configuration  of  the  crew  exploration  vehicle.  Finally,  last  year  we  reported  that 
analysis  of  the  Ares  I  flight  path  also  indicated  that,  under  some  conditions,  the  Ares  I  vehicle  could  hit  the 
launch  tower  during  liftoff  and  the  vehicle  would  need  to  be  steered  away  from  the  launch  tower  or  not 
launched  during  high  winds.  NASA  officials  told  us  they  have  developed  a  plan  to  mitigate  this  risk. 

Project  Office  Comments 

The  Ares  I  project  office  provided  technical  comments  on  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  The  project  office  also  commented  that  it  has  utilized  resources  to  make 
progress  on  the  Constellation  Program  while  focusing  on  goals  that  yield  benefits  to  future  human 
spaceflight  endeavors. 
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Common  Name:  GPM 


Global  Precipitation  Measurement  (GPM)  Mission 


The  Global  Precipitation  Measurement  (GPM) 
mission,  a  joint  NASA  and  Japan  Aerospace 
Exploration  Agency  (JAXA)  project,  seeks  to  improve 
the  scientific  understanding  of  the  global  water  cycle 
and  the  accuracy  of  precipitation  forecasts.  The  GPM 
is  composed  of  a  core  spacecraft  carrying  two  main 
instruments:  a  Dual-frequency  Precipitation  Radar 
(DPR)  and  a  GPM  Microwave  Imager  (GMI).  GPM 
builds  on  the  work  of  the  Tropical  Rainfall  Measuring 
Mission  and  will  provide  an  opportunity  to  calibrate 
measurements  of  global  precipitation. 


Source:  GPM  Project  Office  (artist  depiction). 
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Project  Essentials  Project  Performance 


x  _  ,,  ,  _  (then  year  dollars  in  millions) 

NASA  Center:  Goddard  Space  Flight  Center 


International  Partner:  Japanese  Aerospace  Exploration 

Baseline  Est. 

Latest 

Agency  (JAXA) 

(FY  2009) 

(Feb.  2011) 

Change 

Major  Contractors:  Ball  Aerospace 

Total  Project  Cost 

$975.9 

$928.9 

-4.8% 

Projected  Launch  Date:  July  21 , 2013 

Launch  Location:  Tanegashima  Island,  Japan 

Launch  Vehicle:  JAXA  supplied 

Formulation  Cost 
Development  Cost 
Operations  Cost 

$349.2 

$555.2 

$71.6 

$349.2 

$514.8 

$64.9 

0.0% 

-7.3% 

-9.4% 

Mission  Duration:  3  years 

Launch  Schedule 

7/2013 

7/2013 

0  months 

Project  Summary 

Prior  to  establishing  the  project’s  baseline  cost  and  schedule 
estimate,  NASA  descoped  the  planned  second  spacecraft  of 
the  GPM  mission.  The  project’s  international  partner,  JAXA, 
is  providing  the  launch  vehicle  for  the  core  spacecraft. 
However,  GPM  project  officials  were  tracking  potential 
funding  issues  with  JAXA.  GPM  received  $32  million  under 
the  American  Recovery  and  Reinvestment  Act  of  2009, 
which  was  used  to  maintain  the  current  schedule,  expedite 
some  work  on  the  GMI-1,  and  begin  work  on  a  second  GMI. 


Recent/Continuing  Project 
Challenges 

»-  Funding  Issues 
»-  Development  Partner  Issues 
»-  Design  Issues 

Previously  Reported  Challenges 

»-  Technology  Maturity 
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Common  Name:  GPM 


Global  Precipitation  Measurement  Mission  (GPM) 

Project  Update 

Funding  Issues:  Prior  to  establishing  the  project’s  baseline  cost  estimate,  NASA  removed  funding  for  the 
second  spacecraft  of  the  GPM  mission,  the  Low  Inclination  Observatory  (LIO),  due  to  lack  of  funding. 

The  Low  Inclination  Observatory  (LIO)  was  primarily  intended  to  fly  a  second  GPM  Microwave  Imager 
(GMI-2),  which  would  gather  additional  science  data  to  further  support  the  GPM  mission.  Project  officials 
reported  that  NASA  is  currently  pursuing  an  international  development  partner  willing  to  fund  the  launch 
vehicle  and  spacecraft  needed  for  the  second  GMI  instrument.  However,  despite  de-scoping  the  LIO  launch 
vehicle  and  spacecraft,  the  project  continues  to  invest  in  the  development  of  the  GMI-2  instrument.  A 
GPM  project  official  reported  that  GMI-2  will  be  put  into  storage  in  2013  if  the  LIO  mission  is  not  going  to 
launch  soon  after  that.  Although  the  science  requirements  for  GPM  could  still  be  met  without  flying  the 
GMI-2  instrument,  project  officials  reported  that  without  the  instrument  the  available  science  data  from  the 
mission  would  not  be  as  robust. 

GPM  received  $32  million  under  the  American  Recovery  and  Reinvestment  Act  of  2009.  According  to  project 
officials,  this  enabled  GPM  to  maintain  schedule  in  fiscal  year  2009,  move  some  of  the  GMI  work  planned  for 
fiscal  year  2011  into  fiscal  year  2010,  and  start  the  GMI-2  development  on  schedule  in  October  2009. 

Development  Partner  Issues:  GPM  project  officials  were  tracking  potential  funding  issues  with  the  Japanese 
Aerospace  and  Exploration  Agency  (JAXA),  which  is  providing  the  launch  vehicle  for  the  first  GPM 
spacecraft  as  a  risk  to  the  cost  and  schedule  of  the  project.  In  addition,  the  GPM  project  is  tracking  the 
availability  of  the  JAXA-supplied  Dual-frequency  Precipitation  Radar  (DPR)  instrument.  The  project  reports 
that  delays  in  the  DPR  instrument's  development  have  compressed  the  schedule  available  for  integration 
and  testing. 

Design  Issues:  The  project  has  currently  released  96  percent  of  its  engineering  drawings,  but  only  53 
percent  were  released  by  the  mission  critical  design  review  (CDR)  held  in  December  2009.  A  project  official 
said  that  the  lack  of  released  drawings  at  critical  design  review  didn’t  have  a  serious  impact  in  terms  of 
design  stability  as  testing  was  almost  complete  on  engineering  testing  units  and  flight  units  were  already 
designed  and  ready  to  begin  manufacturing. 

Project  officials  delayed  the  CDR  of  the  fully  demiseable  aluminum  propulsion  tank  from  August  2010  to 
October  2010  due  to  difficulties  with  parts  assembly.  The  GPM  spacecraft  was  designed  to  be  demiseable — 
that  is,  it  will  bum  up  during  re-entry  into  the  Earth’s  atmosphere  to  limit  orbital  debris.  However,  in 
December  2008,  an  updated  re-entry  structural  analysis  at  Johnson  Space  Center  of  GPM  indicated  that  the 
spacecraft  would  not  be  demiseable  as  originally  predicted  by  the  GPM  project  office  and  Johnson  Space 
Center.  The  project  had  initially  delayed  the  start  of  the  implementation  phase  and  establishment  of  GPM 
cost  and  schedule  baselines  by  8  months  in  order  to  reconcile  the  project  budget  with  available  funding  and 
to  resolve  the  demisability  issue. 

Project  Office  Comments 

The  GPM  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  Project  officials  also  commented  that  overall  the  GPM  Project  is  making  progress. 
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Glory 


Glory  project  is  a  low-Earth  orbit  satellite  that  will 
contribute  to  the  U.S.  Climate  Change  Science 
Program.  The  satellite  has  two  principal  science 
objectives:  (1)  collect  data  on  the  properties  of 
aerosols  and  black  carbon  in  the  Earth’s  atmosphere 
and  climate  systems  and  (2)  collect  data  on  solar 
irradiance.  The  satellite  has  two  main  instruments 
— the  Aerosol  Polarimetry  Sensor  (APS)  and  the 
Total  Irradiance  Monitor  (TIM) — as  well  as  two 
cloud  cameras.  The  TIM  will  allow  NASA  to  have 
uninterrupted  solar  irradiance  data  by  bridging 
the  gap  between  NASA’s  Solar  Radiation  and 
Climate  Experiment  and  the  National  Polar-orbiting 
Operational  Environmental  Satellite 
System  (NPOESS). 


Source:  Glory  Project  Office  (artist  depiction). 
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(12/05) 

(7/06) 

(12/10) 

(2/11) 

Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 

International  Partner:  None 

Major  Contractors:  Raytheon  Space  and  Airborne 
Systems,  University  of  Colorado  Laboratory  for  Atmospheric 
and  Space  Physics,  Orbital  Sciences  Corporation 

Projected  Launch  Date:  March  201 1 
Launch  Location:  Vandenberg  AFB,  Calif. 

Launch  Vehicle:  Taurus  XL 

Mission  Duration:  3  years  (5  year  goal) 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2009) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$347.9 

$424.1 

21 .9% 

Formulation  Cost 

$70.5 

$70.8 

0.4% 

Development  Cost* 

$259.1 

$337.6 

30.3% 

Operations  Cost 

$18.3 

$15.8 

-13.7% 

Launch  Schedule 

6/2009 

3/201 1 

21  months 

*Represents  a  1 12%  growth  in  development  costs  since  the 
original  baseline  of  $168.9  established  In  fiscal  year  2008. 


Recent/Continuing  Project 
Challenges 

»-  Launch  Issues 
»-  Funding  Issues 
»-  Parts  Issues 

Previously  Reported  Challenges 

»-  Technology  Maturity 
»-  Complexity  of  Heritage  Technology 
»-  Design  Stability 
»-  Contractor  Performance 


Project  Summary 

Significant  cost  increases  and  schedule  delays  have 
persisted  on  Glory  despite  being  reauthorized  by  Congress 
and  re-baselined  in  2009.  Development  costs  have  increased 
by  about  30  percent  since  2009.  Recent  cost  increases  and 
schedule  delays  are  residual  effects  of  switching  to  an 
alternate  single  board  computer  provider,  the  late  delivery 
of  the  APS  instrument,  and,  more  recently,  due  to  part 
quality  issues  found  in  the  solar  array  drive  assembly. 

Glory  will  launch  on  the  Taurus  XL  launch  vehicle,  which 
is  returning  to  flight  after  the  vehicle  failed  during  a  2009 
launch. 
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Common  Name:  Glory 


Glory 

Project  Update 

Parts  Issues:  The  Glory  project  has  experienced  significant  schedule  delays  due  to  reliability  problems  with 
key  parts  found  during  testing.  For  example,  in  June  2010,  the  project  discovered  excessive  wear  and  debris 
of  the  Slip  Ring  Assembly,  a  part  contained  in  the  solar  array  drive  assembly  that  rendered  one  of  the  array 
wings  unacceptable  for  flight.  The  corrected  solar  array  drive  assembly  was  integrated  with  the  spacecraft 
in  November  2010.  The  other  solar  array  drive  assembly  was  inspected,  found  to  have  no  signs  of  wear  or 
debris,  and  sent  back  to  the  contractor  for  integration  with  the  spacecraft.  This  issue  has  resulted  in  an 
additional  3  month  launch  delay. 

Prior  to  the  solar  array  issue,  the  project  switched  from  using  a  single  board  computer  (SBC)  to  an  alternate 
SBC  produced  by  another  company.  According  to  the  project  manager,  continued  reliability  issues  with  the 
initial  SBC,  including  cracks  in  the  printed  wiring  boards,  required  the  project  to  seek  another  vendor  for 
the  SBC  as  the  part  failed  during  testing.  While  the  new  SBC  has  now  been  integrated  with  the  spacecraft 
and  is  performing  well,  project  officials  estimate  the  total  cost  impact  of  this  switch  in  technology  to  be 
approximately  $60.9  million. 

Launch  Issues:  The  Glory  project  has  been  tracking  the  return  to  flight  activities  of  the  Taurus  XL  launch 
vehicle  as  a  risk  to  achieving  its  launch  readiness  date  in  February  2011.  The  vehicle  failed  during  the  launch 
of  the  Orbiting  Carbon  Observatory  (OCO)  in  February  2009.  The  launch  failure  Mishap  Investigation  Board 
(MIB)  subsequently  released  findings  and  suggested  corrective  actions.  Specifically,  the  MIB  found  that  a 
payload  fairing — a  clamshell-shaped  cover  that  encloses  and  protects  a  payload  during  early  flight — failed 
to  separate  during  ascent.  NASA’s  Launch  Services  Program  has  developed  a  corrective  action  plan  and, 
according  to  a  Launch  Services  Program  official,  the  Taurus  XL  corrective  actions  were  on  track  to  meet  the 
launch  vehicle  readiness  review  for  Glory  in  September  2010.  The  return  to  flight  activities  for  the  Taurus 
XL  is  on-going  while  the  project  performs  test  and  integration  of  instruments  after  the  over  one  year  late 
delivery  of  the  APS  and  a  parts  failure  in  the  Single  Board  Computer.  A  malfunction  in  the  ground  support 
equipment  associated  with  the  Taurus  XL  launch  vehicle  has  subsequently  delayed  the  launch  of  the  Glory 
project  until  March  2011. 

Funding  Issues:  The  Glory  project’s  development  costs  have  increased  by  almost  31  percent  and  its  launch 
has  been  delayed  by  21  months  since  being  reauthorized  by  Congress  and  re-baselined  in  2009  after  a  53 
percent  development  cost  increase.  Cost  increases  and  schedule  delays  are  a  residual  result  of  switching  to 
an  alternate  single  board  computer  provider  due  to  reliability  issues,  the  late  delivery  of  the  APS  instrument, 
and,  more  recently,  due  to  parts  failure  in  the  solar  array  drive  assembly.  Since  Glory’s  original  fiscal  year 
2008  baseline,  the  project’s  development  costs  have  grown  by  113  percent  and  its  launch  has  been  delayed 
over  2  years.  The  Glory  project  also  received  $16  million  under  the  American  Recovery  and  Reinvestment 
Act  of  2009  (ARRA)  which  was  used  to  maintain  the  current  workforce  through  the  planned  launch. 

Project  Office  Comments 

The  Glory  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  also  commented  that  the  project  continues  to  monitor  the 
Taurus  XL  return  to  flight  activities. 
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Common  Name:  GRAIL 


Gravity  Recovery  and  Interior  Laboratory  (GRAIL) 

The  GRAIL  mission  will  seek  to  determine  the 
structure  of  the  lunar  interior  from  crust  to  core, 
advance  our  understanding  of  the  thermal  evolution 
of  the  Moon,  and  extend  our  knowledge  gained  from 
the  Moon  to  other  terrestrial-type  planets.  GRAIL 
will  achieve  its  science  objectives  by  placing  twin 
spacecraft  in  a  low  altitude  and  nearly  circular 
polar  orbit.  The  two  spacecraft  will  perform  high- 
precision  measurements  between  them.  Analysis  of 
changes  in  the  spacecraft-to-spacecraft  data  caused 
by  gravitational  differences  will  provide  direct  and 
precise  measurements  of  lunar  gravity.  GRAIL  will 
ultimately  provide  a  global,  high-accuracy,  high- 
resolution  gravity  map  of  the  Moon. 

Source:  Courtesy  of  NASA/JPL-Caltech  (artist  depiction). 
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(12/07) 
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Preliminary 
design  review 
(11/08) 


A 

Project 

confirmation 

(1/09) 


Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory  (JPL) 

International  Partners:  None 

Major  Contractors:  Lockheed  Martin 

Projected  Launch  Date:  September  8,  201 1 
Launch  Location:  Kennedy  Space  Center,  Fla. 
Launch  Vehicle:  Delta  II  Heavy 

Mission  Duration:  9  months 
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A 

Critical  design 

GAO 

Launch 

review 

review 

readiness  date 

(11/09) 

(12/10) 

(9/11) 

Project  Performance 

(then  year  dollars 

in  millions) 

Baseline  Est. 

Latest 

(FY  2009) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$496.2 

$496.2 

0.0% 

Formulation  Cost 

$50.5 

$50.5 

0.0% 

Development  Cost 

$427.0 

$427.0 

0.0% 

Operations  Cost 

$18.7 

$18.7 

0.0% 

Launch  Schedule 

9/201 1 

9/2011 

0  months 

Recent/Continuing  Project 
Challenges 

»-  Technology  Issues 
»-  Launch  Issues 


Project  Summary 

During  formulation  it  was  determined  that  the  reaction 
wheel  assembly  did  not  meet  mission  requirements.  The 
project  office  undertook  a  new  development  effort  of 
the  reaction  wheel,  but  because  of  a  mechanical  design 
flaw  found  in  testing,  it  will  not  be  delivered  on  schedule. 
In  addition,  the  schedule  for  testing  and  integration  for 
avionics  has  been  impacted  by  late  delivery  of  parts 
and  hardware  problems.  Project  officials  continue  to  be 
concerned  about  the  availability  of  Delta  II  Heavy  launch 
personnel  and  resources  for  the  mission. 
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Gravity  Recovery  and  Interior  Laboratory  (GRAIL) 

Project  Update 

Technology  Issues:  GRAIL  project  officials  said  they  included  no  new  technology  in  designing  the  GRAIL 
orbiters  to  keep  the  mission  simple,  cost  effective,  and  as  close  to  the  Gravity  Recovery  and  Climate 
Experiment  (GRACE)  mission  as  possible.  Therefore,  the  GRAIL  project  instruments  are  similar  to  those 
used  in  the  GRACE  mission.  All  heritage  technologies  for  the  project,  except  for  the  reaction  wheel 
assembly,  were  deemed  mature  at  the  preliminary  design  review.  Project  officials  told  us  that  during 
formulation  they  reviewed  the  reaction  wheel  assembly  and  determined  that  it  did  not  meet  the  standards 
for  this  mission  and  caused  the  project  to  undertake  a  new  development  effort.  The  electronics  of  the 
newly  developed  reaction  wheel  are  combined  into  the  mechanical  assembly,  and  the  project  decreased  the 
diameter  of  the  mechanical  assembly.  However,  the  reaction  wheel  assembly  flight  units  are  not  on  track 
for  on-time  delivery  because  of  a  mechanical  design  flaw  found  in  testing.  The  project  determined  that  there 
was  a  problem  with  the  bearing  material  and  modifications  had  to  be  made  to  allow  for  proper  load  bearings 
and  stability.  The  project  has  determined  the  root  cause  of  the  problem  and  developed  a  design  update  to 
correct  the  problem.  Project  officials  said  that  schedule  contains  enough  margin  to  accommodate  the  late 
delivery  of  the  reaction  wheel  assembly  without  affecting  the  launch  schedule. 

Launch  Issues:  Last  year,  we  reported  that  GRAIL  project  officials  were  concerned  about  the  availability 
of  trained  personnel  to  process  the  launch  since  GRAIL  would  have  been  the  last  NASA  project  to  launch 
on  the  Delta  II  launch  vehicle.  Since  that  time,  the  NPOESS  Preparatory  Project  (NPP)  has  delayed  its 
launch  date,  and  therefore,  GRAIL  is  no  longer  the  last  NASA  project  scheduled  to  launch  on  the  Delta 
II  launch  vehicle.  Project  officials  told  us  they  continue  to  be  concerned  about  the  availability  of  Delta  II 
launch  personnel  and  continue  to  monitor  that  availability  as  a  risk  to  the  project.  NASA  launch  services  is 
monitoring  changes  in  Delta  II  launch  services  personnel  and  processes  and  the  post-production  support 
proposals  for  all  major  subcontractors. 

Other  Issues  to  be  Monitored:  Project  officials  told  us  the  delivery  of  the  avionics  flight  boxes  have  been 
delayed  due  to  late  delivery  of  parts,  which  will  impact  the  system  level  environmental  tests  for  these 
units  and,  therefore,  are  on  the  critical  path.  However,  the  project  mitigated  this  risk  by  using  engineering 
test  units  of  the  avionics  boxes  since  the  flight  unit  deliveries  were  delayed  past  the  beginning  of  test  and 
integration  in  July  2010.  Project  officials  told  us  that  the  project  can  conduct  system-level  testing  using 
engineering  test  units  if  the  avionics  boxes  are  further  delayed  since  the  electronics  boards  are  the  same  in 
both  units  and  can  be  swapped  out  prior  to  the  system-level  environmental  testing.  The  project  expects  that 
the  two  flight  units  will  be  delivered  by  early  2011.  The  project  has  modified  its  schedule  to  accommodate 
for  the  delay  in  the  delivery  of  the  flight  avionics  and  reported  it  has  sufficient  schedule  margin  to  meet  the 
launch  date. 

Project  Office  Comments 

The  GRAIL  project  office  commented  that  the  project  has  completed  all  the  major  milestones  on  schedule 
and  is  currently  on  track  to  meet  its  launch  readiness  date. 
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Ice,  Cloud,  and  Land  Elevation  Satellite-2  (ICESat-2) 

NASA’s  ICESat-2  is  a  first-tier  mission  recommended 
by  the  National  Research  Council  in  its  2007  Earth 
Science  Decadal  Survey.  ICESat-2  is  a  follow-on 
mission  to  ICE  Sat,  tasked  with  measuring  changes 
in  polar  ice-sheet  mass  with  space-borne  altimetry 
measurements  to  understand  mechanisms  that  drive 
change  and  the  impact  of  these  changes  on  future 
global  sea  level.  ICESat-2  will  be  utilizing  a  micro¬ 
pulse  multi-beam  laser  instrument  with  a  photon 
counting  approach  to  measurement.  This  process 
will  allow  for  dense  cross-track  sampling  with  a  high 
repetition  rate,  allowing  ICESat-2  to  provide  better 
elevation  estimates  over  high  slope  and  rough  areas. 

Source:  ICESat-2  Project  Office  (artist  depiction). 
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Preliminary  Critical  design 

design  review  review 

(11/11)  (11/12) 
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Launch 

readiness  date 
(10/15) 

Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 

International  Partner:  None 

Major  Contractors:  none 

Projected  Launch  Date:  October  2015 
Launch  Location:  Vandenberg  AFB,  Calif. 

Launch  Vehicle:  TBD 

Mission  Duration:  5  years 


Project  Performance 
(then  year  dollars  in  millions) 

Latest 
(Feb.  2011) 

Preliminary  Estimate  of 

Project  Life  Cycle  Cost*  not  available 

*  The  project  has  not  yet  reached  the  point  in  the  acquisition 
life  cycle  where  a  preliminary  life  cycle  cost  estimate  would 
normally  be  developed. 


Launch  Schedule  10/2015 


Recent  Project  Challenges 

Launch  Issues 
»-  Funding  Issues 


Project  Summary 

ICESat-2  was  approved  to  begin  formulation  in 
December  2009.  The  project’s  internal  cost  estimates 
exceeded  the  cost  cap,  which  led  the  project  to  evaluate 
potential  cost  reduction  activities  and  re-scope  options. 
These  activities  delayed  the  Mission  Definition  Review 
originally  planned  for  August  2010  until  January  2011. 
The  project  used  $20.4  million  in  American  Recovery  and 
Reinvestment  Act  of  2009  funds  to  work  with  four  major 
laser  vendors  to  mature  the  micro-pulse  laser  designs. 
However,  the  acquisition  and  testing  for  the  laser 
subsystem  is  behind  schedule. 
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Ice,  Cloud,  and  Land  Elevation  Satellite-2  (ICESat-2) 

Project  Update 

Launch  Issues :  ICESat-2  is  tracking  a  risk  due  to  the  lack  of  medium  class  launch  vehicle  availability. 

The  project  is  concerned  that  a  delay  in  identifying  a  launch  vehicle  for  the  mission  will  lead  to  cost  and 
schedule  impact.  The  only  certified  vehicle  currently  available  to  NASA  missions  in  the  ICESat-2  launch 
time  frame  is  the  Atlas  V,  an  intermediate  launch  vehicle.  The  only  medium  class  launch  vehicle  currently 
available  under  NASA’s  contract  for  launch  services  is  the  Falcon  9;  however,  it  has  not  yet  been  certified. 

If  ICESat-2  selects  the  Falcon  9,  the  mission  launch  date  would  be  tied  to  a  successful  certification  of  the 
launch  vehicle.  The  Atlas  V  comes  at  a  higher  cost  than  what  NASA  has  traditionally  paid  for  a  medium 
capability  launch  vehicle.  Officials  told  us  that  the  project  is  currently  allocating  $100  million  for  the  launch 
vehicle.  The  project  planned  to  develop  a  procurement  package  to  initiate  procurement  of  a  launch  vehicle 
in  early  fiscal  year  2011. 

Funding  Issues :  NASA  provided  cost  parameters  for  the  ICESat-2  mission;  however,  the  project’s  internal 
life  cycle  cost  estimates  exceeded  the  cost  cap  by  $100  million.  Project  officials  are  currently  evaluating  how 
they  can  reduce  the  project’s  life-cycle  cost  estimates  through  various  re-scoping  options,  such  as  partnering 
with  another  ongoing  mission  or  reducing  the  mission  life.  Due  to  these  activities,  the  project’s  Mission 
Definition  Review,  originally  scheduled  for  August  2010,  was  not  scheduled  to  occur  until  January  2011  at 
the  earliest.  In  addition,  the  project  used  $20.4  million  in  American  Recovery  and  Reinvestment  Act  of  2009 
(ARRA)  funding  for  the  micro-pulse  laser  development  contracts  to  retire  project  risk  earlier.  However,  the 
acquisition  and  testing  of  these  laser  subsystems  is  behind  schedule  due  to  delays  associated  with  the  ARRA 
reporting  by  the  agency.  Also,  according  to  project  officials,  the  project  received  $28  million  in  fiscal  year 
2010  funding  from  the  President’s  global  climate  initiative,  but  it  was  unable  to  use  all  of  the  additional  funds 
within  the  fiscal  year  and  is  unsure  whether  it  will  receive  funding  from  this  initiative  in  fiscal  year  2011. 

Other  Issues  to  be  Monitored :  The  project  entered  the  formulation  phase  in  December  2009.  During  the 
mission  concept  review  process,  the  project  responded  to  changing  science  requirements,  particularly 
the  need  to  accurately  measure  slope  through  micro-pulse  laser  technology.  The  Advanced  Topographic 
Laser  Altimeter  System  is  the  single  instrument  on  the  ICESat-2  mission.  The  project  identified  two  critical 
technologies,  the  micro-pulse  lasers  and  the  Laser  Reference  System  (LRS).  The  project  expects  that  both 
technologies  will  be  mature  at  the  preliminary  design  review  scheduled  for  November  2011.  The  micro¬ 
pulse  lasers  being  developed  for  ICESat-2  use  a  low  energy  pulse  at  a  high  frequency,  a  change  from  the  high 
power  lasers  used  on  the  original  ICESat  mission.  The  project  is  working  with  four  major  laser  vendors  to 
mature  the  micro-pulse  laser  technology  and  designs.  Despite  delays  in  awarding  the  contracts,  the  vendors 
are  working  toward  the  original  milestone  delivery  dates  to  reduce  schedule  risk.  The  LRS  is  designed  to 
provide  absolute  laser  pointing  knowledge  in  order  to  pinpoint  the  ice  footprint  location  6  meters  on  the 
ground. 

Project  Office  Comments 

The  ICESat-2  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  Project  officials  also  commented  that  ICESat-2  is  currently  in  formulation  and  activities  are 
on-going  to  confirm  a  mission  that  fits  within  the  cost  cap.  NASA  does  not  formally  commit  to  a  project’s 
schedule  and  cost  until  Key  Decision  Point  (KDP)-C,  which  ICESat-2  has  not  yet  reached. 
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James  Webb  Space  Telescope  (JWST) 


The  James  Webb  Space  Telescope  (JWST)  is  a 
large,  infrared-optimized  space  telescope  that  is 
designed  to  find  the  first  galaxies  that  formed  in 
the  early  universe.  Its  focus  will  include  searching 
for  first  light,  assembly  of  galaxies,  origins  of  stars 
and  planetary  systems,  and  origins  of  the  elements 
necessary  for  life.  JWST’s  instruments  will  be 
designed  to  work  primarily  in  the  infrared  range  of 
the  electromagnetic  spectrum,  with  some  capability 
in  the  visible  range.  JWST  will  have  a  large  mirror, 

6.5  meters  (21.3  feet)  in  diameter  and  a  sunshield  the 
size  of  a  tennis  court.  Both  the  mirror  and  sunshade 
will  not  fit  onto  the  rocket  fully  open,  so  both  will 
fold  up  and  open  once  JWST  is  in  outer  space.  JWST 
will  reside  in  an  orbit  about  1.5  million  kilometers  (1 
million  miles)  from  the  Earth. 


Source:  Northrop  Grumman  Aerospace  Systems  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 
International  Partners:  European  Space  Agency  (ESA), 
Canadian  Space  Agency  (CSA) 

Major  Contractors:  Northrop  Grumman 

Projected  Launch  Date:  June  2014 
Launch  Location:  Kourou,  French  Guiana 
Launch  Vehicle:  Ariane  5  (ESA  Supplied) 

Mission  Duration:  5  years  (1 0  year  goal) 


Project  Performance 


(then  year  dollars 

in  millions) 

Baseline  Est. 

Latest 

(FY  2009) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$4963.6 

$5095.4 

2.7% 

Formulation  Cost 

$1800.1 

$1800.2 

0.0% 

Development  Cost 

$2581.1 

$2710.9 

5.0% 

Operations  Cost 

$582.4 

$584.5 

0.4% 

Launch  Schedule 

6/2014 

6/2014 

0  months 

Recent/Continuing  Project 
Challenges 

»-  Funding  Issues 
»-  Contractor  Issues 
»-  Design  Issues 

Previously  Reported  Challenges 

»-  Complexity  of  Heritage  Technology 


Project  Summary 

NASA  is  taking  steps  to  address  deficiencies  identified  by 
two  independent  reviews  this  year.  One  independent  review 
panel  found  that  the  earliest  possible  launch  date  for  JWST 
is  September  2015,  a  15-month  delay  from  the  baseline 
estimate.  To  meet  this  date,  the  panel  estimated  the  project 
would  need  an  additional  $500  million  over  the  next  2 
fiscal  years  and  a  total  life-cycle  cost  of  approximately  $6.5 
billion.  A  separate  review  team  reported  that  JWST’s  test 
plans  exceeded  the  money  and  time  available.  As  a  result  of 
these  reviews,  the  program  office  at  NASA  headquarters  will 
now  report  directly  to  the  NASA  Associate  Administrator. 
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James  Webb  Space  Telescope  (JWST) 

Project  Update 

Funding  Issues:  According  to  an  October  2010  Independent  Comprehensive  Review  Panel  (ICRP)  report, 
JWST’s  baseline  did  not  reflect  the  most  probable  cost  and  resulted  in  a  project  that  was  not  executable  with 
the  given  budget.  The  ICRP  found  that  the  budget  was  understated  because  it  did  not  include  known  threats 
and  provided  insufficient  reserves,  particularly  in  the  year  of  confirmation  and  the  year  following.  The 
panel  also  reported  problems  with  overall  project  management  and  a  lack  of  effective  oversight  by  Division 
managers  who  concurred  with  the  project’s  practice  of  deferring  work  to  later  years  without  assessing 
the  future  impact.  To  address  existing  funding  concerns,  JWST  received  $75  million  under  the  American 
Recovery  and  Reinvestment  Act  of  2009.  Despite  these  additional  funds,  the  ICRP  found  that  the  earliest 
launch  date  possible  is  September  2015 — 15  months  after  the  baseline  schedule.  Further,  the  ICRP  reported 
that  JWST’s  life-cycle  cost  would  likely  increase  by  $1.4  billion  or  more,  $500  million  of  which  would 
be  required  in  the  next  2  fiscal  years.  In  response  to  the  panel’s  recommendations,  NASA  made  several 
organizational  changes,  including  establishing  a  new  program  office  at  headquarters  that  reports  directly  to 
the  NASA  Associate  Administrator  and  managing  the  project’s  budget  separately  from  Astrophysics. 

Contractor  Issues:  At  confirmation,  the  project  believed  it  had  sufficient  insight  into  contractor  performance 
to  predict  future  trends  and  used  Earned  Value  Management  data  to  predict  cost  overruns  at  the  contractor. 
Project  officials  told  us  that  shortly  after  confirmation  the  prime  contractor  and  a  subcontractor  came 
forward  with  previously  unidentified  risks  to  project  cost,  leaving  the  project  with  insufficient  reserves. 

The  ICRP  found  that  the  project  had  identified  these  cost  risks,  but  failed  to  account  for  them  in  project 
reserves  because  they  had  not  yet  been  formally  documented  by  the  contractor.  The  project  intends  to  take 
over  testing  and  integration  responsibilities  for  the  OTE/ISIM  instruments  from  the  contractor.  Despite  these 
challenges,  the  project  is  approaching  the  end  of  the  5-year  polishing  phase  for  the  OTE  primary  mirror 
segments  and  started  the  fourth  round  of  cryo  testing  on  the  primary  mirrors  in  May  2010. 

Design  Issues:  The  project  has  identified  challenges  in  analytically  demonstrating  that  the  design  of  the 
ISIM  composite  structure  had  the  necessary  strength  and  performance  capability.  The  ISIM  structure  and 
the  bonds  used  to  attach  instruments  must  be  designed  to  withstand  very  low  temperatures  for  an  indefinite 
period.  The  project  needed  to  develop  and  verify  new  analytical  techniques  for  testing  which  required 
additional  time  and  money.  At  mission  critical  design  review,  the  project  planned  for  two  thermal  and  optical 
performance  tests  of  the  ISIM.  The  project  continues  to  track  ISIM’s  thermal  testing  as  a  major  risk. 

Other  Issues  to  be  Monitored:  The  scale,  complexity,  and  cryogenic  nature  of  JWST  prohibit  a  traditional 
“Test  as  you  Fly”  end-to-end  testing  program;  therefore,  the  project  is  more  dependent  on  analysis  and 
subcomponent  testing.  After  the  mission  critical  design  review,  NASA  chartered  a  Test  Assessment  Team 
(TAT)  to  evaluate  the  project’s  test  plans.  The  TAT  report  found  that  some  of  the  test  plans  exceeded  the 
money  and  time  available  and  made  recommendations  to  prioritize  verification  tasks,  help  the  project  gain 
efficiencies,  particularly  in  the  thermal  testing,  and  reduce  costs  and  shorten  the  schedule.  The  project  has 
formally  concurred  with  most  of  the  TAT  recommendations.  The  project  also  addressed  residual  concerns 
from  the  mission  preliminary  design  review  over  the  sunshield  testing  at  the  instrument  CDR  in  January 
2010  and  is  pending  closure  as  the  project  works  on  details  of  the  test  plan. 

Project  Office  Comments 

The  JWST  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  The  project  officials  also  commented  that  the  project  and  its  international 
partners  have  made  good  technical  progress  and  retired  some  of  the  highest  technical  risks.  In  addition, 
NASA  is  executing  a  reorganization  of  the  project  and  developing  a  new  independent  cost  estimate  to 
address  management  and  budget  challenges  highlighted  in  the  recent  ICRP  report. 
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Juno 


The  Juno  mission  seeks  to  improve  our 
understanding  of  the  origin  and  evolution  of  Jupiter. 
Juno  plans  to  achieve  its  scientific  objectives  by 
using  a  simple,  solar-powered  spacecraft  to  make 
global  maps  of  the  gravity,  magnetic  fields,  and 
atmospheric  conditions  of  Jupiter  from  a  unique 
elliptical  orbit.  The  spacecraft  carries  precise, 
highly  sensitive  radiometers,  magnetometers,  and 
gravity  science  systems.  Juno  is  slated  to  make  32 
orbits  to  sample  Jupiter’s  full  range  of  latitudes 
and  longitudes.  From  its  polar  perspective,  Juno 
is  designed  to  combine  local  and  remote  sensing 
observations  to  explore  the  polar  magnetosphere  and 
determine  what  drives  Jupiter’s  auroras. 


Source:  NASA/JPL  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory  (JPL) 
International  Partners:  Agenda  Spaziale  Italiana  (ASI)- 
Selex  Galileo,  ASI  -  Thales  Alenia  Space,  Centre  Spatial  de 
Liege  Belgian  Science  Policy,  Centre  National  d'Etudes 
Spatiales  -  Centre  d'Etude 

Major  Contractors:  Lockheed  Martin 

Projected  Launch  Date:  August  5,  201 1 
Launch  Location:  Cape  Canaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Mission  Duration:  6  years 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2009) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$1107.0 

$1107.0 

0.0% 

Formulation  Cost 

$186.3 

$186.3 

0.0% 

Development  Cost 

$742.3 

$742.3 

0.0% 

Operations  Cost 

$178.4 

$178.4 

0.0% 

Launch  Schedule 

8/201 1 

8/201 1 

0  months 

Recent/Continuing  Project 
Challenges 

»-  Technology  Issues 
»-  Design  Issues 
»-  Parts  Issues 
»-  Contractor  Issues 

Previously  Reported  Challenges 

»-  Development  Partner  Issues 


Project  Summary 

Juno  continues  to  address  issues  with  heritage  technology. 
The  Command  and  Data  Handling  Unit,  a  required 
component  of  the  spacecraft,  remains  on  the  critical  path 
due  to  late  workforce  ramp-up  by  the  contractor  and  start 
of  the  flight  design  effort  and  could  cause  a  delay  in  the 
scheduled  launch.  Furthermore,  modifications  have  been 
made  to  the  Command  and  Data  Handling  Unit’s  Module 
Interface  Card  board  to  address  Mars  Reconnaissance 
Orbiter  in  flight  issues.  Finally,  poor  materials  quality 
caused  the  failure  of  certain  components  of  the 
spacecraft’s  solar  arrays  during  testing  and  led  to  a  change 
in  supplier. 
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Juno 


Project  Update 

Technology  Issues:  After  the  preliminary  design  review,  the  project  reassessed  the  Toroidal  Low  Gain 
Antenna  (TLGA)  as  being  immature  when  it  was  determined  that  the  materials  being  used  in  the  highly 
charged  particle  environment  could  store  an  electrical  charge,  which  would  in  turn  interfere  with  some 
lower-level  science  requirements  from  two  of  the  instruments  on  the  spacecraft.  The  project  has  since 
coated  the  surface  of  the  TLGA  with  germanium  to  provide  a  discharge  path  to  the  grounded  metal  structure 
that  resolved  the  interference  issue. 

Design  Issues:  The  Juno  project  had  released  only  39  percent  of  the  engineering  drawings  at  the  critical 
design  review  (CDR).  Project  officials,  however,  said  they  used  engineering  models  for  all  instruments  to 
demonstrate  design  maturity  at  CDR.  For  some  spacecraft  components,  the  Juno  project  did  not  build  or 
test  engineering  models  because  they  were  of  heritage  designs.  For  example,  some  spacecraft  components 
being  utilized  are  very  similar  to  the  ones  used  on  the  Mars  Reconnaissance  Orbiter  (MRO);  therefore,  the 
project  accepted  some  of  the  spacecraft  card  designs  based  on  qualification  testing.  In  addition,  subsystem 
and  component-level  reviews  were  held  prior  to  the  mission  CDR,  and  project  officials  told  us  the  results 
of  these  lower-level  reviews  provided  evidence  that  the  design  was  stable.  However,  modifications  have 
been  made  to  the  Command  and  Data  Handling  Unit’s  Module  Interface  Card  (CMIC)  board  to  respond  to 
two  series  of  reset/sideswap  events  found  during  the  MRO  design  review  as  well  as  MRO  in-flight  software 
issues.  The  root  cause  of  the  problems  in  the  MRO  CMIC  board  has  not  been  determined,  but  Juno  has  made 
a  total  of  12  design  changes  to  mitigate  the  problems  in  Juno’s  CMIC  design. 

Parts  Issues/Contractor  Issues:  The  molybdenum  tabs,  parts  attached  to  the  solar  cells  used  to  conduct 
power  from  the  cells  to  the  solar  array  power  harness,  failed  during  testing.  The  project  established  a 
failure  review  board  that  found  the  failures  were  caused  by  poor  materials  quality.  The  project  subsequently 
switched  the  material  supplier  for  this  part.  The  failure  review  board  also  investigated  solar  array  disbonding 
issues  and  found  that  they  were  caused  by  contractor  workmanship  errors  in  the  surface  preparation  of  the 
solar  array  panels.  The  contractor  adjusted  its  procedures  and  re-fabricated  the  panels. 

Other  Issues  to  be  Monitored:  Juno  project  officials  said  that  they  began  integration  and  testing  in  April 
2010.  The  project  is  experiencing  delays  in  the  delivery  of  the  Command  and  Data  Handling  (C&DH)  module 
as  a  result  of  late  workforce  ramp-up  and  a  late  start  of  the  flight  design  effort.  The  C&DH  module  remains 
on  the  critical  path  and  could  cause  a  delay  to  Juno’s  launch.  Assembly  and  testing  has  begun  with  a  test 
unit  version  of  the  C&DH  module  while  design  issues  are  addressed  on  the  flight  unit.  Furthermore,  to 
address  schedule  concerns  on  the  Italian  Space  Agency’s  (ASI)  development  of  the  Ka-band  translator  after 
the  2009  earthquake  in  Italy,  the  project  requested  and  ASI  agreed  to  upgrade  the  engineering  model  to  be 
a  flyable  engineering  model.  This  flyable  engineering  model  has  already  been  fully  tested,  delivered  to  the 
Juno  project,  and  installed  on  the  flight  system.  Although  the  project  expected  to  fly  the  engineering  model, 
work  continued  on  the  original  flight  model.  The  original  flight  model  was  delivered  and  integrated  on  the 
spacecraft  in  September  2010. 

Project  Office  Comments 

The  Juno  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  Project  officials  also  commented  that  the  project  has  successfully  resolved  several  technical 
issues  and  has  accommodated  any  delays  via  technical  and  schedule  resiliency  and  that  the  project  team 
continues  to  make  good  progress  toward  its  projected  launch  date  of  August  5,  2011. 
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Landsat  Data  Continuity  Mission  (LDCM) 

The  Landsat  Data  Continuity  Mission  (LDCM),  a 
partnership  between  NASA  and  the  U.S.  Geological 
Survey,  seeks  to  extend  the  ability  to  detect  and 
quantitatively  characterize  changes  on  the  global 
land  surface  at  a  scale  where  natural  and  man-made 
causes  of  change  can  be  detected  and  differentiated. 

It  is  the  successor  mission  to  Landsat  7.  The  Landsat 
data  series,  begun  in  1972,  is  the  longest  continuous 
record  of  changes  in  the  Earth’s  surface  as  seen 
from  space.  Landsat  data  is  a  resource  for  people 
who  work  in  agriculture,  geology,  forestry,  regional 
planning,  education,  mapping,  and  global  change 
research. 


Source:  Orbital  (artist  depiction). 
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Latest 

(FY  2010) 

(Feb.  2011) 

Change 

Major  Contractors:  Ball  Aerospace  and  Technologies 

Corp.,  Orbital  Sciences  Corp.,  The  Hammers  Company 

Total  Project  Cost 

$941.7 

$941 .6 

0.0% 

Projected  Launch  Date:  June  2013 

Launch  Location:  Vandenberg  AFB,  Calif. 

Launch  Vehicle:  Atlas  V 

Formulation  Cost 
Development  Cost 
Operations  Cost 

$341.5 

$583.4 

$16.8 

$341 .4 
$587.6 
$12.5 

0.0% 

0.7% 

-25.6% 

Mission  Duration:  5  years  (10  years  propellant) 

Launch  Schedule 

6/2013 

6/2013 

0  months 

Project  Summary 

In  December  2009,  NASA  established  a  baseline  launch 
readiness  date  for  the  LDCM  project  of  June  2013.  However, 
internally  the  project  continues  to  plan  for  a  December  2012 
launch  in  order  to  avoid  or  minimize  a  gap  in  LANDSAT 
data.  When  the  project  established  the  baseline,  the 
Thermal  Infrared  Sensor  (TIRS)  instrument  was  officially 
added  to  the  scope  of  the  mission,  increasing  the  mission 
cost  by  approximately  $160  million.  The  project  is  tracking 
parts  issues  for  all  of  its  major  components — the  TIRS 
and  the  Operational  Land  Imager  instruments  and  the 
spacecraft.  The  cost  and  schedule  impacts  of  some  of  these 
issues  are  uncertain. 


Recent  Project  Challenges 

»-  Funding  Issues 
»-  Parts  Issues 

Previously  Reported  Challenges 

»-  Technology  Maturity 
»-  Development  Partner  Performance 
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Landsat  Data  Continuity  Mission  (LDCM) 

Project  Update 

Funding  Issues:  Last  year  the  project  reported  an  estimated  lifecycle  cost  range  of  $730-800  million  but 
established  a  baseline  life-cycle  cost  estimate  of  $941.7  million  due  to  the  addition  of  the  Thermal  Infrared 
Sensor  (TIRS)  instrument  in  December  2009,  at  an  estimated  additional  cost  of  $160  million.  The  TIRS 
instrument  was  officially  added  to  the  scope  of  LDCM  due  to  demand  from  the  science  community.  With  that 
addition,  LDCM’s  instrument  payload  consists  of  two  instruments,  the  Operational  Land  Imager  (OLI) — a 
multi-spectral  imaging  sensor  to  detect  and  characterize  land  changes — and  the  TIRS — a  sensor  that  has 
a  wide  range  of  uses,  including  water  resource  management  and  wildfire  risk  assessment.  LDCM  received 
$63.4  million  in  American  Recovery  and  Reinvestment  Act  (ARRA)  funding,  and  used  the  money  to  procure 
items  for  the  components  of  the  TIRS  instrument,  the  spacecraft,  and  the  OLI  instrument. 

At  confirmation  in  December  2009,  the  project  and  the  Standing  Review  Board  presented  Joint  Cost  and 
Schedule  Confidence  Level  (JCL)  results  based  on  mutually  agreeable  risks  and  uncertainty  factors.  The 
JCL  estimates  developed  for  the  project  resulted  in  a  50-percent  confidence  level  launch  date  of  December 
2012,  and  a  70-percent  confidence  date  of  June  2013.  The  project  continues  to  plan  internally  for  a  December 
2012  launch  date  in  order  to  avoid  a  potential  data  gap  and  has  $91  million  budgeted  for  risk  mitigation  in 
order  to  meet  the  earlier  date.  LDCM  is  working  with  its  ground  system  partner,  the  United  States  Geological 
Survey  (USGS),  to  determine  the  likelihood  of  a  data  availability  gap  and  steps  to  mitigate  the  risk  of  a 
gap.  Additionally,  to  address  funding  shortfalls  at  USGS  and  reduce  the  risk  to  mission  success,  NASA  and 
USGS  amended  the  final  implementation  agreement  for  LDCM  to  increase  NASA’s  role  in  the  ground  system 
development  and  shift  some  of  the  funding  responsibilities  to  USGS  in  later  years,  which  decreased  the 
LDCM  estimate  for  operations  to  decrease  by  25  percent. 

Parts  Issues:  The  project  is  tracking  risks  associated  with  the  TIRS  and  OLI  instruments  and  the  spacecraft. 
The  project  discovered  that  the  main  electronics  boards  on  the  main  electronics  box  of  the  TIRS  instrument 
were  not  meeting  thermal  stability  requirements.  While  TIRS  is  a  new,  in-house  development  effort  and  is  on 
the  project’s  critical  path,  many  of  the  subsystems  and  components  were  used  in  earlier  flight  projects.  The 
issues  with  the  main  electronics  box  cost  $3.8  million,  but  the  problem  had  no  net  impact  to  the  project’s 
schedule.  The  OLI  instrument  experienced  problems  with  the  black  chrome  plating  and  dark  mirror  coating. 
According  to  project  officials,  the  black  chrome  plating  did  not  withstand  testing  and  lost  adhesion,  due  to 
poor  plating  processes  at  the  vendor.  As  a  result,  the  vendor  rebuilt  the  Solar  Calibration  Assembly.  These 
issues  currently  have  no  overall  impact  on  the  project’s  schedule,  and  the  cost  impacts  have  been  negotiated. 
On  the  spacecraft,  the  project  identified  contamination  of  the  Reaction  Wheel  Assembly  (RWA)  lubricant 
and  scheduled  to  have  new  bearings  installed  by  the  vendor.  Project  officials  said  that  they  have  identified 
windows  during  integration  and  test  where  a  new  unit  can  be  inserted.  Although  the  problem  caused  a  six 
month  schedule  slip  for  the  RWA,  the  project  expects  no  impact  on  the  overall  schedule  because  the  delay 
was  largely  absorbed  by  the  integration  and  testing  workarounds  and  subsystem  schedule  slack. 

Last  year,  we  reported  that  the  project  had  released  83  percent  of  its  design  drawings  as  of  September  2009. 
In  April  2010,  the  project  had  released  93  percent  of  its  drawings  and  held  a  successful  mission  critical 
design  review  (CDR)  in  May  2010,  but  the  project  is  tracking  risks  on  each  of  the  major  components. 
Currently,  the  project  reports  that  97  percent  of  the  total  design  drawings  have  been  released. 

Project  Office  Comments 

The  LDCM  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated  as 
appropriate.  Project  officials  also  commented  that  the  mission  has  set  a  commitment  for  a  launch  readiness 
date  of  June  2013,  but  the  project  is  aggressively  working  to  launch  in  December  2012  in  order  to  minimize 
the  chance  of  a  data  gap  should  Landsat  5  or  Landsat  7  cease  operations. 


Page  54 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


Common  Name:  LADEE 


Lunar  Atmosphere  and  Dust  Environment  Explorer  (LADEE) 


The  Lunar  Atmosphere  and  Dust  Environment 
Explorer  (LADEE)  mission  objective  is  to  determine 
the  global  density,  composition,  and  time  variability 
of  the  lunar  atmosphere.  LADEE ’s  measurements  will 
determine  the  size,  charge,  and  spatial  distribution  of 
electrostatically  transported  dust  grains.  Additionally, 
LADEE  will  carry  an  optical  laser  communications 
demonstrator  that  will  test  high-bandwidth 
communication  from  lunar  orbit. 


Source:  LADEE  Project  Office  (artist  depiction). 
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Project  Essentials  Project  Performance 

(then  year  dollars  in  millions) 

NASA  Center  Lead:  Ames  Research  Center 


International  Partners:  None 

Baseline  Est. 
(FY  2010) 

Latest 
(Feb.  2011) 

Change 

Major  Contractors:  None 

Total  Project  Cost* 
Formulation  Cost 

$262.9 

$79.5 

$262.9 

$79.5 

0.0% 

0.0% 

Projected  Launch  Date:  November  2013 

Development  Cost 

$168.2 

$168.2 

0.0% 

Launch  Location:  Wallops  Flight  Facility,  Va. 

Launch  Vehicle:  Minotaur  V 

Operations  Cost 

$15.2 

$15.2 

0.0% 

Mission  Duration:  180  days 

Launch  Schedule 

11/2013 

11/2013 

0  months 

*This  estimate  does  not  include  the  LLCD  instrument  which 
is  being  funded  by  the  Space  Operations  Mission  Directorate 
at  a  cost  of  approximately  $65  million. 


Recent  Project  Challenges 

»-  Technology  Issues 
»-  Parts  Issues 
»-  Launch  Issues 


Project  Summary 

The  LADEE  project  was  confirmed  on  August  23,  2010,  to 
proceed  into  implementation.  LADEE  will  be  flying  three 
heritage  instruments,  as  well  as  the  Lunar  Laser  Com 
Demo,  which  is  being  developed  by  the  Space  Operations 
Mission  Directorate  at  a  cost  of  approximately  $65 
million.  NASA  will  launch  the  project  on  the  Minotaur 
V.  A  bid  protest  delayed  the  issuance  of  a  delivery  order 
for  the  launch  vehicle  and  postponed  development  of  a 
Soft-Ride  system  that  will  protect  instrumentation  during 
launch. 
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Lunar  Atmosphere  and  Dust  Environment  Explorer  (LADEE) 

Project  Update 

Technology  Issues:  LADEE  utilizes  three  instruments  that  have  been  designed  for  other  missions  but  require 
modifications  to  their  form,  fit,  and  function.  None  of  the  three  instruments  were  considered  mature  at 
the  preliminary  design  review  in  July  2010.  NASA  flew  the  Lunar  Dust  Experiment  (LDEX)  on  various 
configurations  on  the  HEOS  2,  Galileo,  Ulysses,  and  Cassini  projects.  The  Neutral  Mass  Spectrometer  (NMS) 
is  a  subset  of  the  Sample  Analysis  at  Mars  instrument  being  developed  for  the  Mars  Science  Laboratory.  The 
Ultraviolet  Spectrometer  (UVS)  is  based  on  the  design  of  the  UVS  instrument  flown  on  the  Lunar  Crater 
Observation  and  Sensing  Satellite  (LCROSS).  The  project  will  also  fly  the  Lunar  Laser  Com  Demo  (LLCD) 
as  a  ride  along  technology  demonstration  on  LADEE.  The  LLCD  is  being  developed  by  the  Space  Operations 
Mission  Directorate  at  a  cost  of  approximately  $65  million,  which  is  not  included  in  the  LADEE  cost 
estimates. 

Parts  Issues:  The  UVS  has  run  into  problems  with  the  source  vendor  and  parts  quality  and,  therefore,  is  not 
identical  to  the  LCROSS  version  of  the  instrument.  Project  officials  determined  that  the  printed  wiring  board 
for  the  UVS  was  being  developed  in  a  facility  with  no  quality  systems  or  workmanship  standards  in  place. 
The  project  decided  to  keep  the  printed  wiring  board  design,  but  had  another  vendor  produce  the  boards  at 
a  NASA-approved  facility.  Implementation  of  this  change  cost  the  project  approximately  $1.1  million. 

Launch  Issues:  LADEE  will  be  launched  on  a  Minotaur  V,  which  was  procured  under  the  Air  Force’s 
indefinite  delivery  indefinite  quantity  contract  with  a  commercial  launch  vehicle  provider.  A  bid  protest 
regarding  the  selection  of  the  Minotaur  V,  however,  delayed  the  issuance  of  the  delivery  order  for  the 
vehicle  and  the  project’s  preliminary  design  review  by  3  months  and  the  critical  design  review  by  5  months. 
Furthermore,  the  project  will  need  to  equip  the  launch  vehicle  with  a  Soft-Ride  system  in  order  to  protect 
the  project’s  instrumentation  from  excessive  vibration  during  launch.  While  there  is  no  new  development 
effort  behind  the  Soft-Ride,  the  system  must  be  tuned  to  the  particular  load  environment  and  spacecraft 
design,  which  will  be  delayed  until  the  launch  vehicle  delivery  order  is  issued. 

Other  Issues  to  be  Monitored:  The  LADEE  project  has  not  reached  a  design  review  where  we  could  assess 
design  stability.  As  of  September  2010,  the  project  expected  to  release  58  percent  of  its  design  drawings  by 
the  preliminary  design  review  and  83  percent  by  the  critical  design  review. 

Because  of  its  focus  on  being  a  low  cost  mission,  LADEE’s  only  critical  technology  is  the  RF  antenna  on  the 
spacecraft,  which,  according  to  the  project  office,  is  proceeding  on  schedule. 

Project  Office  Comments 

The  LADEE  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  LADEE  project  officials  also  commented  that  the  bid  protest  on  the  launch 
vehicle  has  been  resolved  and  that  the  Minotaur  will  be  procured  under  an  Air  Force  contract  with  a 
commercial  launch  service  provider. 
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Magnetospheric  Multiscale  (MMS) 

The  Magnetospheric  Multiscale  (MMS)  is  made  up  of 
four  identically  instrumented  spacecraft.  The  mission 
will  use  the  Earth's  magnetosphere  as  a  laboratory 
to  study  the  microphysics  of  magnetic  reconnection, 
energetic  particle  acceleration,  and  turbulence. 

Magnetic  reconnection  is  the  primary  process  by 
which  energy  is  transferred  from  solar  wind  to 
Earth’s  magnetosphere  and  is  the  physical  process 
determining  the  size  of  a  space  weather  storm.  The 
spacecrafts  will  fly  in  a  pyramid  formation,  adjustable 
over  a  range  of  10  to  400  kilometers,  enabling  them 
to  capture  the  three-dimensional  structure  of  the 
reconnection  sites  they  encounter.  The  data  from 
MMS  will  be  used  as  a  basis  for  predictive  models  of 
space  weather  in  support  of  exploration. 

Source:  MMS  Project  Office  (Computer  Model). 
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Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 
International  Partners:  Austria,  France,  Japan,  Sweden 

Major  Contractors:  Southwest  Research  Institute 

Projected  Launch  Date:  March  2015 
Launch  Location:  Cape  Canaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Mission  Duration:  2  years 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2010) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$1082.7 

$1082.7 

0.0% 

Formulation  Cost 

$173.0 

$173.0 

0.0% 

Development  Cost 

$857.4 

$857.4 

0.0% 

Operations  Cost 

$52.3 

$52.3 

0.0% 

Launch  Schedule 

3/2015 

3/2015 

0  months 

Recent  Project  Challenges 

»-  Development  Partner  Issues 
»-  Design  Issues 
»-  Technology  Issues 


Project  Summary 

The  MMS  project  used  $6  million  in  cost  reserves  to 
move  development  work  for  the  Spin  Plane  Double 
Probe  instrument  from  Sweden  to  the  University  of 
New  Hampshire  because  Sweden  was  not  providing 
adequate  levels  of  funding  for  project  development.  The 
movement  of  development  work  has  resulted  in  a  delay 
of  approximately  6  months  for  the  completion  of  the 
design  for  the  instrument.  However,  project  officials  do 
not  believe  the  delay  will  impact  the  mission’s  March 
2015  launch  readiness  date. 
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Magnetospheric  Multiscale  (MMS) 

Project  Update 

Development  Partner  Issues:  The  MMS  project  used  approximately  $6  million  in  reserve  funds  to  move 
work  from  Sweden  to  the  University  of  New  Hampshire  because  Sweden  was  not  making  satisfactory 
progress  on  the  production  of  the  Spin  Plane  Double  Probe  (SDP)  instrument  due  to  inadequate  levels 
of  funding.  After  considering  three  potential  candidates,  the  MMS  project  selected  the  University  of  New 
Hampshire  in  2010  to  assume  production  of  the  SDP  deployment  mechanism,  the  most  complex  element 
of  the  SDP  instrument.  Sweden  will  continue  to  provide  SDP  flight  hardware  as  well  as  mission  science 
support.  As  a  result  of  these  changes,  the  completion  of  the  design  for  the  SDP  is  behind  schedule  by 
approximately  6  months,  but  MMS  officials  believe  this  change  poses  no  threat  to  the  mission’s  launch 
readiness  date  in  March  2015. 

Design  Issues:  In  August  2010,  the  project  completed  its  mission  critical  design  review  (CDR).  At  that 
time  the  project  had  released  77  percent  of  its  engineering  design  drawings.  Last  year,  project  officials 
told  us  that  having  70  to  80  percent  of  design  drawings  completed  by  CDR  is  normal,  but  they  had  not 
established  any  goals  for  the  project.  MMS  officials  stated  that  the  number  of  complete  engineering  test 
units  is  as  important,  if  not  more  so,  than  design  drawings.  According  to  project  officials,  MMS  uses  high 
fidelity  instrument  models  as  a  risk  reduction  effort.  By  using  engineering  models  that  are  as  flight-ready 
as  possible,  project  officials  reported  that  they  can  see  where  problems  are  and  better  identify  risks. 
Additionally,  they  stated  that  proceeding  with  the  manufacture  of  flight  hardware  without  having  built  flight¬ 
like  engineering  units  to  test  the  design,  will  almost  always  lead  to  schedule  overruns  to  solve  design  issues. 

Technology  Issues:  Following  mission  CDR  in  August  2010,  the  MMS  project  has  yet  to  fully  address  the 
form,  fit,  and  function  of  the  payload  separation  system,  a  key  heritage  technology.  All  four  MMS  satellites 
will  launch  stacked  on  a  single  Atlas  V  launch  vehicle.  When  the  top  spacecraft  deploys,  springs  will  push 
off  the  first  satellite  and  trigger  a  command  for  each  subsequent  satellite  to  deploy.  The  technology  required 
for  the  separation  system  is  not  new;  however,  the  project  is  working  closely  with  the  contractor  to  ensure 
that  all  four  satellites  separate  in  a  consistent  manner  which  supports  the  need  for  them  to  fly  in  a  pyramid 
formation. 

Other  Issues  to  be  Monitored:  MMS  was  authorized  to  enter  formulation,  the  phase  that  precedes 
implementation,  in  2002  with  an  initial  cost  estimate  of  $369  million.  The  project  was  authorized  to  enter 
implementation  in  June  2009  with  a  baseline  life-cycle  cost  estimate  of  over  $1  billion.  The  project  manager 
said  the  initial  cost  estimate  was  for  a  smaller  instrument  suite  than  what  is  currently  planned  for  the 
mission  and  added  that  one  cost  driver  for  the  project  since  the  initial  cost  estimate  was  the  requirement  for 
magnetic  and  electrostatic  cleanliness.  The  initial  cost  estimate  also  did  not  account  for  the  higher  cost  of 
the  Atlas  V,  which  is  a  larger  launch  vehicle  than  the  Delta  II  initially  considered  by  the  project. 

Project  Office  Comments 

The  MMS  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  Project  officials  also  commented  that  MMS  continues  to  make  technical  progress.  In  2010, 
the  MMS  project  completed  the  detailed  design  of  the  instruments  and  spacecraft. 
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Mars  Atmosphere  and  Volatile  EvolutioN  (MAVEN) 

The  Mars  Atmosphere  and  Volatile  EvolutioN 
(MAVEN)  mission,  a  robotic  orbiter  mission,  will 
provide  a  comprehensive  picture  of  the  Mars  upper 
atmosphere,  ionosphere,  solar  energetic  drivers, 
and  atmospheric  losses.  MAVEN  will  deliver 
comprehensive  answers  to  long-standing  questions 
regarding  the  loss  of  Mars’  atmosphere,  climate 
history,  liquid  water,  and  habitability.  MAVEN  will 
provide  the  first  direct  measurements  ever  taken 
to  address  key  scientific  questions  about  Mars’ 
evolution. 


Source:  NASA  GSFC  MAVEN  Project  Office  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 
International  Partners:  Centre  D'Etude  Spatiale  des 
Rayonnements,  Toulouse,  France 

Major  Contractors:  Lockheed  Martin,  University  of 
California  Space  Science  Laboratory,  University  of 
Colorado  Laboratory  for  Atmospheric  and  Space  Physics 

Projected  Launch  Date:  November  18,  2013 
Launch  Location:  Cape  Canaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Mission  Duration:  1  year  of  travel,  1  year  of  operations 
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Recent  Project  Challenges 

»-  Design  Issues 
»-  Launch  Issues 


Project  Summary 

MAVEN  was  selected  under  the  Mars  Scout  Program — a 
NASA  initiative  to  send  a  series  of  small,  low-cost  robotic 
missions  to  Mars.  The  project  was  competitively  selected 
from  innovative  proposals  by  the  scientific  community. 
The  project  is  relying  on  heritage  technologies,  but 
project  officials  acknowledged  that  these  technologies 
required  modifications  to  their  form,  fit,  and  function 
to  operate  as  necessary  for  MAVEN’s  requirements.  The 
project  is  being  designed  to  the  Atlas  V  launch  vehicle, 
which  is  significantly  more  expensive  than  it  was  under 
the  previous  launch  services  contract. 


Page  59 


GAO-11-239SP  Assessments  of  Selected  Large-Scale  Projects 


Common  Name:  MAVEN 


Mars  Atmosphere  and  Volatile  EvolutioN  (MAVEN) 

Project  Update 

Design  Issues:  At  the  preliminary  design  review,  the  project  manager  decided  not  to  authorize  the  Respin 
of  the  High  Efficiency  Power  Supply  (HEPS),  MAVEN’s  power  supply  system,  because  of  a  high  probability 
of  failure  and  therefore  violates  the  mission  assurance  requirements.  The  project  met  with  the  contractor 
to  discuss  HEPS  design,  fabrication,  assembly,  test  history  and  qualification  in  order  to  resolve  this  issue. 

The  MAVEN  project  has  not  reached  a  design  review  where  we  could  assess  design  stability.  At  the  mission 
preliminary  design  review  in  July  2010,  the  project  estimated  that  it  would  have  85  percent  of  its  engineering 
drawings  released  at  the  critical  design  review. 

Launch  Issues:  According  to  project  officials,  the  project  was  given  approval  to  initiate  selection  of  a  launch 
vehicle  in  September  2010  after  the  new  NASA  Launch  Services  (NLS)  contract  was  awarded.  Project 
officials  told  us  the  project  had  been  designing  to  two  vehicles  prior  to  the  new  contract  being  awarded. 
However,  the  only  available  vehicle  that  currently  meets  the  needs  of  the  MAVEN  project  is  the  intermediate- 
class  Atlas  V,  which  will  be  significantly  more  expensive  than  it  was  under  the  previous  NLS  contract.  In 
October  2010,  NASA  announced  that  the  Atlas  V  had  been  selected  as  the  launch  vehicle  for  MAVEN  at  a 
total  cost  of  $187  million.  Science  Mission  Directorate  officials  told  us  that  they  incorporated  this  increased 
cost  into  the  project’s  baseline  during  the  confirmation  review. 

Other  Issues  to  be  Monitored:  In  order  to  control  project  costs,  the  project  plans  to  minimize  development 
activities  of  new  technology  by  designing  MAVEN  spacecraft  and  instruments  based  on  available  heritage 
hardware.  The  MAVEN  project  identified  seven  heritage  technologies,  all  of  which  are  required  to  meet 
the  mission’s  science  requirements.  Prior  to  the  preliminaiy  design  review,  the  project  deemed  all  heritage 
technologies  to  be  mature,  but  project  officials  acknowledged  that  these  heritage  technologies  do  not  take 
into  account  modifications  of  form,  fit,  and  function  needed  to  operate  in  the  Martian  environment  and 
require  modifications.  For  example,  while  MAVEN’s  magnetometer  design  is  similar  to  those  flown  on  prior 
NASA  projects,  a  minor  change  to  the  electronics  of  the  magnetometer  is  necessary  to  extend  its  dynamic 
range.  The  project  is  also  concerned  that  measurements  from  the  magnetometer  may  become  corrupted 
due  to  the  amount  of  electronic  interference,  or  noise,  on  the  spacecraft.  To  alleviate  this  concern,  project 
officials  decided  to  reconfigure  the  solar  cells  on  the  panel  to  minimize  the  magnetic  field  at  the  location 
of  the  instrument.  As  a  result  of  this  reconfiguration  and  additional  analysis,  project  officials  reported  the 
risk  has  been  mitigated.  Furthermore,  project  officials  told  us  they  are  evaluating  ways  to  ensure  that  the 
spacecraft  and  instruments  will  continue  to  operate  and  collect  data  during  major  solar  flares. 

Project  Office  Comments 

The  MAVEN  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  also  commented  that  the  project  entered  into  implementation 
in  October  2010  and  is  on  track  for  critical  design  review  scheduled  for  July  2011. 
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Mars  Science  Laboratory  (MSL) 

The  Mars  Science  Laboratory  (MSL)  is  part  of  the 
Mars  Exploration  Program  (MEP).  The  MEP  seeks 
to  understand  whether  Mars  was,  is,  or  can  be  a 
habitable  world.  To  answer  this  question,  the  MSL 
project  will  investigate  how  geologic,  climatic, 
and  other  processes  have  worked  to  shape  Mars 
and  its  environment  over  time,  as  well  as  how  they 
interact  today.  The  MSL  will  continue  this  systematic 
exploration  by  placing  a  mobile  science  laboratory 
on  the  Mars  surface  to  assess  a  local  site  as  a 
potential  habitat  for  life,  past  or  present.  The  MSL  is 
considered  one  of  NASA’s  flagship  projects  and  will 
be  the  most  advanced  rover  yet  sent  to  explore  the 
surface  of  Mars. 

Source:  NASA/JPL-Caltech. 
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Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory  (JPL) 
Partners:  U.S.  Department  of  Energy,  Centre  Nationale 
d'Etude  Spatiale  (France),  Russian  Federal  Space  Agency, 
Centro  de  Astrobiologia  (Spain),  Canadian  Space  Agency 

Major  Contractors:  In-house  development 

Projected  Launch  Date:  November  2011 
Launch  Location:  Cape  Canaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Mission  Duration:  1  year  of  travel,  2  years  of  operations 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2010) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$2394.2 

$2476.3 

3.4% 

Formulation  Cost 

$515.5 

$515.5 

0.0% 

Development  Cost* 

$1719.9 

$1802.0 

4.8% 

Operations  Cost 

$158.8 

$158.8 

0.0% 

Launch  Schedule 

11/2011 

11/2011 

0  months 

*Represents  an  86%  growth  in  development  costs  since  the 
original  baseline  of  $968.6  established  in  fiscal  year  2008. 


Recent/Continuing  Project 
Challenges 

»-  Design  Issues 
»-  Parts  Issues 

Challenges  Previously  Reported 

»-  Technology  Maturity 
»-  Complexity  of  Heritage  Technology 


Project  Summary 

Congress  reauthorized  the  MSL  and  it  was  subsequently 
re-baselined  in  January  2010  because  the  project  had 
exceeded  its  2008  cost  baseline  by  more  than  30  percent. 
In  2009,  MSL’s  cost  had  grown  more  than  $834  million  and 
its  scheduled  launch  had  been  delayed  26  months  from  its 
original  2008  baseline  due  to  work  needed  to  overcome 
technical  challenges  with  the  actuators  and  avionics.  This 
increase  includes  more  than  an  86  percent  increase  in 
development  costs. 
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Common  Name:  MSL 


Mars  Science  Laboratory  (MSL) 

Project  Update 

Congress  reauthorized  the  MSL  in  the  Consolidated  Appropriations  Act  of  2010  and  NASA  subsequently  re¬ 
baselined  the  project  in  January  2010  after  it  had  exceeded  its  2008  development  cost  baseline  by  more  than 
30  percent.  Since  the  original  project  baseline  in  2008,  the  life-cycle  cost  for  the  project  has  increased  by 
more  than  $834  million — including  more  than  an  86  percent  increase  in  development  costs — and  the  launch 
has  been  delayed  until  November  2011  since  launch  windows  for  Mars  mission  are  optimally  aligned  every 
26  months.  These  cost  and  schedule  overruns  were  driven  by  problems  with  the  actuators  and  avionics. 
Specifically,  the  project  experienced  problems  with  the  actuators  that  allow  the  vehicle  to  move  and  execute 
the  sample  operations  performed  by  the  lab.  The  project  has  since  redesigned  the  actuators  and  retired  this 
risk.  The  project  indicated  that  project  reserves  may  be  inadequate  to  meet  the  scheduled  work  for  2011. 

Design  Issues:  The  MSL  project  design  was  not  stable  at  the  Critical  Design  Review  (CDR).  Several  design 
changes  were  required  after  CDR  to  address  various  issues.  For  example,  project  officials  told  us  the 
avionics  hardware  was  a  new  design  and  had  been  delivered  in  an  immature  state.  They  had  hoped  to  have 
all  issues  with  the  avionics  hardware  completed  by  November  2009;  however,  project  officials  said  the 
design  of  the  hardware  is  still  not  complete  and  the  project  has  delayed  the  software  development  which 
includes  about  12  deliverables.  The  avionics  computer  element  is  currently  the  leading  risk  to  the  MSL 
schedule  and  its  functionality  is  critical  to  the  mission’s  success. 

Furthermore,  the  Sample  Analysis  at  Mars  Wide  Range  Pump  has  had  a  series  of  development  problems 
and  although  the  project  has  worked  through  about  10  engineering  models,  it  continues  to  struggle  to 
pass  the  life  test.  The  project  built  and  tested  two  different  pump  designs  in  parallel  that  met  the  science 
requirements  and  conducted  an  accelerated  life  test  on  them.  The  project  plans  to  make  a  decision  between 
the  two  designs  at  the  conclusion  of  the  life  test  and  pump  qualification  testing,  currently  scheduled  for 
fall  2010.  The  project  is  also  monitoring  performance  degradation  of  the  Multi  Mission  Radioisotope 
Thermoelectric  Generator  (MMRTG)  due  to  the  thermocouples  that  convert  the  heat  generated  by  the 
plutonium  into  electricity  degrading  at  a  faster  rate  than  predetermined,  or  about  10  percent.  According  to 
the  project  manager,  the  MMRTG  can  still  meet  its  objectives  with  a  10  percent  decay  rate,  but  if  this  rate 
increases  the  project  cannot  meet  its  requirements  and  will  be  forced  to  cut  the  nominal  number  of  samples 
collected  or  the  distance  the  rover  is  to  travel  during  the  primary  mission. 

Parts  Issues:  The  project  experienced  a  parts  failure  associated  with  the  transition  joints  in  the  propulsion 
system  which  caused  the  joints  to  fail  under  load.  Project  officials  reported  this  issue  was  realized  after  the 
project  finished  building  its  propulsion  system,  causing  the  project  to  rebuild  the  system  and  adopt  a  new 
joint  design.  The  transition  to  the  new  design  required  a  rework  and  retest  of  the  descent  cruise  stages. 
According  to  project  officials,  the  project  also  encountered  parts  issues  on  the  avionics  package,  including 
a  shorting  out  of  the  pins  on  the  avionics  processor  and  a  packaging  issue  that  caused  a  disconnect  between 
the  analog  components  and  the  configuration  of  the  board. 

Project  Office  Comments 

The  MSL  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  The  project  believes  that  the  GAO  assessment  largely  reflects  the  history  of  the  project  and 
most  of  the  issues  identified  have  been  resolved. 
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Common  Name:  NPP 


NPOESS  Preparatory  Project  (NPP) 

The  National  Polar-orbiting  Operational 
Environmental  Satellite  System  (NPOESS) 

Preparatory  Project  (NPP)  is  a  joint  mission  with  the 
National  Oceanic  and  Atmospheric  Administration 
(NOAA)  and  the  U.S.  Air  Force.  The  satellite  will 
measure  ozone,  atmospheric  and  sea  surface 
temperatures,  land  and  ocean  biological  productivity, 

Earth  radiation,  and  cloud  and  aerosol  properties. 

The  NPP  mission  has  two  objectives.  First,  NPP 
will  provide  a  continuation  of  global  weather 
observations  following  the  Earth  Observing  System 
missions  Terra  and  Aqua.  Second,  NPP  will  function 
as  an  operational  satellite  and  will  provide  data  until 
the  first  NPOESS  satellite  launches. 

Source:  Ball  Aerospace. 
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Project  Essentials  Project  Performance 


~  ..  .  o  ,  (then  year  dollars  in  millions) 

NASA  Center  Lead:  Goddard  Space  Flight  Center 


Partner:  National  Oceanic  and  Atmospheric 

Baseline  Est. 

Latest 

Administration  and  U.S.  Air  Force 

(FY  2007) 

(Feb.  2011) 

Change 

Major  Contractors:  Northrop  Grumman  Electrical 

Systems  and  Ball  Aerospace  and  Technologies  Corp. 

Total  Project  Cost 
Formulation  Cost 
Development  Cost* 

$672.8 

$47.3 

$593.0 

$864.3 

$47.1 

$780.1 

28.5% 
0.5% 
31 .6% 

Projected  Launch  Date:  October  201 1 

Launch  Location:  Vandenberg  AFB,  Calif. 

Operations  Cost 

$32.5 

$37.1 

14.0% 

Launch  Vehicle:  Delta  II 

Launch  Schedule 

4/2008 

10/2011 

42  months 

Mission  Duration:  5  years  *Represents  a  52.1%  growth  in  development  costs  since  the 

original  baseline  of  $513  million  established  in  fiscal  year  2004. 

Project  Summary 

NPP  has  experienced  over  $183  million  in  development  cost 
growth  and  a  42-month  launch  delay,  and  officials  told  us 
that  there  is  more  work  remaining  than  the  schedule  allows. 
The  last  of  the  partner-provided  instruments  was  delivered 
for  integration  on  the  satellite  in  June  2010,  although  a 
number  of  risks  remain.  Project  officials  said  that  many 
problems  were  uncovered  late  in  the  development  process, 
leading  NASA  to  revise  NPP  mission  success  criteria.  In 
February  2010,  the  White  House  announced  a  restructuring 
of  the  NPOESS  program,  which  could  affect  the  launch 
schedule. 


Recent/Continuing  Project 
Challenges 

»-  Development  Partner  Issues 
»-  Launch  Issues 

Previously  Reported  Challenges 

»-  Technology  Maturity 
»-  Complexity  of  Heritage  Technology 
»-  Design  Stability 
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Common  Name:  NPP 


NPOESS  Preparatory  Project  (NPP) 

Project  Update 

NPP  project  officials  have  attributed  cost  and  schedule  overruns  to  development  partner  challenges  and 
a  lack  of  central  authority  between  the  three  NPOESS  agencies.  Further,  DOD,  with  agreement  from  its 
partner  agencies,  restructured  the  NPOESS  program  in  2006,  but  the  program  continued  to  experience 
cost  and  schedule  growth.  Since  NPP  was  baselined  in  fiscal  year  2007,  the  project’s  development  cost  has 
increased  by  26  percent  in  the  fiscal  year  2011  budget  request,  and  its  schedule  has  increased  by  42  months. 

Development  Partner  Issues:  Management  and  developmental  partner  challenges  have  continued  to  result 
in  cost  overruns  and  schedule  delays  in  the  Visible  Infrared  Imaging  Radiometer  Suite  (VIIRS)  and  Cross¬ 
track  Infrared  Sounder  (CrIS)  instruments.  The  project  office  attributes  almost  all  of  the  cost  and  schedule 
changes  to  the  late  delivery  of  these  partner-provided  instruments.  The  CrIS  was  the  last  instrument  to  arrive 
for  NPP  and  was  delivered  to  the  spacecraft  contractor  in  June  2010.  Issues  with  the  CrIS  instrument  moved 
the  launch  date  from  January  2011  to  October  2011.  Furthermore,  because  NPOESS  is  now  not  scheduled  to 
launch  until  2014,  NPP  will  still  be  a  demonstration  satellite  as  originally  intended  but  will  have  to  function 
as  an  operational  satellite,  providing  interim  data  until  NPOESS  launches. 

In  February  2010,  the  White  House  announced  plans  to  restructure  the  NPOESS  program,  into  the  Joint 
Polar  Satellite  System  (JPSS),  to  address  cost  overruns  and  schedule  delays.  As  a  result  of  the  restructure, 
NOAA  and  DOD  will  undertake  separate  satellite  system  acquisitions.  The  NPOESS  program  continues 
to  develop  the  instruments  and  ground  systems  supporting  NPP,  but,  according  to  project  officials,  the 
management  of  the  instruments’  contracts  is  being  transferred  from  the  NPOESS  Integrated  Program 
Office  (IPO),  which  is  a  joint  U.S.  Air  Force  and  NOAA  program  office,  to  DOD’s  Space  and  Missile  Systems 
Center.  The  NPP  project  is  taking  steps  to  facilitate  cooperation  and  gain  more  authority  with  the  technical 
elements  than  it  had  at  the  beginning  of  NPP  but  believes  the  restructuring  will  cause  further  launch  delays 
due  to  fiscal  constraints  stemming  from  a  lack  of  necessary  funds  to  cover  termination  liability  for  NPOESS 
contracts. 

Although  all  critical  technologies  are  mature,  NPP  continues  to  report  an  inability  to  reduce  risks  to  an 
acceptable  level  on  three  instruments  provided  by  its  development  partners — the  VIIRS,  the  CrIS,  and  the 
Ozone  Mapper  Profiler  Suite.  Project  officials  told  us  they  lack  confidence  in  the  processes  used  by  the  IPO, 
are  unsure  how  these  instruments  will  function  on  orbit.  Further,  they  believe  there  is  more  work  remaining 
than  the  schedule  allows  for  an  October  2011  launch.  For  example,  the  NPP  project  is  currently  tracking  the 
VIIRS  system’s  door  deployment  testing  as  a  schedule  risk.  Because  of  the  uncertainty  of  the  instrument’s 
functionality,  NASA  is  updating  the  NPP  Mission  Success  Criteria  based  on  these  risk  assessments  in  order 
to  lower  expectations  and  define  minimum  mission  success  criteria. 

Launch  Issues:  Since  this  will  be  one  of  the  last  missions  to  be  launched  on  a  Delta  II,  NASA  is  tracking 
the  availability  of  trained  personnel  to  launch  NPP  as  a  risk.  While  NASA  rates  the  impact  of  a  launch  slip 
on  NPP  and  the  other  three  remaining  missions  scheduled  for  the  Delta  II  as  high  risk,  the  agency  currently 
considers  this  as  a  low  probability  as  there  are  sufficient  existing  processes  and  mitigation  efforts  in  place. 

Project  Office  Comments 

The  NPP  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated  as 
appropriate.  Project  officials  also  commented  the  project  is  working  with  the  newly  formed  JPSS  Program 
to  finalize  an  integrated  NPP  schedule  to  launch.  They  added  that  NPP  will  continue  to  be  a  demonstration 
satellite  for  NPOESS/JPSS.  However,  with  the  NPOESS/JPSS-1  satellite’s  launch  delay  to  2014,  agencies  will 
use  the  NPP  data  operationally. 
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Orbiting  Carbon  Observatory  2  (OCO-2) 

NASA’s  Orbiting  Carbon  Observatory  2  (OCO-2)  is 
based  on  the  original  OCO  mission  that  failed  to 
reach  orbit  in  2009  and  is  designed  to  enable  more 
reliable  predictions  of  climate  change.  It  will  make 
precise,  time-dependent  global  measurements  of 
atmospheric  carbon  dioxide.  These  measurements 
will  be  combined  with  data  from  a  ground-based 
network  to  provide  scientists  with  the  information 
needed  to  better  understand  the  processes  that 
regulate  atmospheric  carbon  dioxide  and  its 
role  in  the  carbon  cycle.  NASA  hopes  enhanced 
understanding  of  the  carbon  cycle  will  improve 
predictions  of  future  atmospheric  carbon  dioxide 
increases  and  the  potential  impact  on  the  climate. 

Source:  Jet  Propulsion  Laboratory  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory 

International  Partner:  None 

Major  Contractors:  Orbital  Sciences  Corp. 

Projected  Launch  Date:  February  2013 
Launch  Location:  Vandenberg  AFB,  Calif. 
Launch  Vehicle:  Taurus  XL 

Mission  Duration:  2  years 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2010) 

(Feb.  2011) 

Change 

Total  Project  Cost 

$349.9 

$349.9 

0.0% 

Formulation  Cost 

$60.9 

$60.9 

0.0% 

Development  Cost 

$249.0 

$249.0 

0.0% 

Operations  Cost 

$40.0 

$40.0 

0.0% 

Launch  Schedule 

2/2013 

2/2013 

0  months 

Recent  Project  Challenges 

»-  Parts  Issues 
»-  Funding  Issues 


Project  Summary 

OCO-2  entered  a  tailored  formulation  phase  in  March  2010. 
The  project  management’s  goal  is  to  minimize  changes 
from  the  OCO  mission.  The  project  office  worked  with 
NASA  to  develop  preliminary  cost  estimates,  which  are 
higher  than  the  2008  estimate  of  $273.1  million  for  OCO, 
due  in  part  to  the  project  obtaining  a  full  set  of  spares  for 
OCO-2.  NASA  has  selected  the  Taurus  XL  launch  vehicle 
for  OCO-2,  the  same  vehicle  used  for  the  OCO  mission.  The 
project  received  $18  million  under  the  American  Recovery 
and  Reinvestment  Act  of  2009  that  was  used  to  enable  the 
earliest  possible  launch. 
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Orbiting  Carbon  Observatory  2  (OCO-2) 

Project  Update 

Parts  Issues:  The  project  is  making  every  effort  to  duplicate  the  original  OCO  design  using  identical 
hardware,  drawings,  documents,  procedures,  and  software  wherever  possible  and  practical  in  order  to 
produce  OCO-2  with  minimum  cost,  schedule,  and  performance  risk.  However,  project  officials  stated  that 
there  were  no  engineering  models  for  many  of  the  OCO  components  and  the  original  components  were 
lost  on  OCO,  making  the  rebuild  difficult,  particularly  due  to  obsolescence  of  parts.  The  OCO-2  project  will 
procure  a  full  set  of  spares  to  help  avoid  problems  with  parts  obsolescence  during  the  development  and 
testing  of  flight  hardware.  OCO-2  encountered  difficulties  with  two  particular  components  due  to  lack  of 
spares  and  parts  obsolescence.  The  cryocooler  used  on  OCO  was  a  spare  that  the  project  received  at  no 
cost;  however,  the  same  cryocoolers  were  not  available  for  OCO-2.  Additionally,  the  flight  computer  from 
OCO  is  now  obsolete.  OCO-2  is  redesigning  and  updating  the  flight  computer  in  order  to  avoid  converting 
all  technology  to  a  new  flight  computer.  Project  officials  said  they  held  a  successful  critical  design  review 
(CDR)  for  the  redesigned  flight  computer  based  on  an  engineering  development  unit  and  they  expected  the 
new  design  to  be  fully  validated  by  the  end  of  2010. 

Funding  Issues:  The  OCO-2  project  office  helped  NASA  develop  a  life-cycle  cost  estimate  based  on  the 
original  life-cycle  costs  of  OCO.  In  December  2008,  OCO’s  life-cycle  cost  estimate  was  $273.1  million, 
compared  to  OCO-2’s  baseline  estimate  of  $349.9  million.  Project  officials  attributed  the  higher  life-cycle 
cost  estimate  for  OCO-2  to  development  of  a  new  crycooler,  inflation,  procurement  of  a  full  set  of  spares, 
and  an  increase  in  the  cost  of  the  launch  vehicle.  For  example,  NASA  did  not  have  acquisition  costs  for  the 
cryocooler  for  the  original  OCO  mission.  OCO-2  is  acquiring  two  new  cryocoolers  through  an  interagency 
transfer  with  the  National  Oceanic  and  Atmospheric  Administration  (NOAA),  but  will  have  to  contract 
for  two  new  units  to  provide  to  NOAA  for  its  future  use.  The  project  also  used  $18  million  under  the 
American  Recovery  and  Reinvestment  Act  of  2009  to  acquire  long  lead  items  for  the  spacecraft,  instrument 
development,  and  project  management  to  enable  the  earliest  possible  launch  of  an  OCO  recovery  mission. 

Other  Issues  to  be  Monitored:  OCO-2  entered  a  tailored  formulation  phase  in  March  2010  to  expedite 
entering  implementation  because  the  project  has  been  designed  and  built  once.  According  to  project 
officials  the  tailored  formulation  reduces  the  number  of  reviews;  therefore,  OCO-2’s  first  major  review  was 
the  mission  CDR,  which  was  held  in  August  2010,  and  preceded  project  confirmation.  At  CDR,  the  project 
had  released  95  percent  of  its  engineering  drawings  for  the  instrument  and  spacecraft.  In  June  2010,  NASA 
selected  Orbital  Sciences  Corporation  to  launch  OCO-2  aboard  a  Taurus  XL,  the  same  vehicle  used  for  OCO 
in  2009.  Orbital  and  NASA  ran  concurrent  mishap  investigations  following  the  OCO  launch  failure,  and 
Orbital  has  addressed  the  findings  of  each  report.  The  Glory  mission,  the  first  to  launch  on  the  Taurus  XL 
since  the  2009  launch  failure,  is  scheduled  to  launch  in  March  2011.  OCO-2  is  the  next  mission  in  line  for  the 
Taurus  XL. 

OCO-2  includes  a  single  instrument,  the  three-channel  grating  spectrometer,  based  on  heritage  technology 
from  the  OCO  mission.  Although  the  project  reports  that  the  spectrometer’s  technology  maturity  is  high,  the 
project  will  make  minor  changes  in  components  and  some  obsolete  parts  that  will  need  to  be  replaced. 

Project  Office  Comments 

The  OCO-2  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated  as 
appropriate.  The  project  officials  also  commented  that  OCO-2  is  intended  to  duplicate,  as  much  as  possible, 
the  OCO  mission  that  was  lost  due  to  the  Taurus  XL  failure.  As  such,  OCO-2  was  granted  a  waiver  from  the 
normal  NASA  project  formulation  process.  OCO-2  is  baselining  a  launch  in  February  2013. 
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Orion  Crew  Exploration  Vehicle 

NASA’s  Orion  Crew  Exploration  Vehicle,  was 
designed  to  carry  crew  and  cargo  to  the  International 
Space  Station  (ISS)  and  to  the  Moon  as  part  of  the 
Constellation  Program.  The  5-meter  diameter  Orion 
capsule  was  designed  to  be  launched  by  the  Ares  I 
Crew  Launch  Vehicle  and  to  carry  four  astronauts  to 
the  ISS  and  the  Moon  after  linking  up  with  an  earth 
departure  stage.  The  capsule  will  return  to  Earth 
and  descend  on  parachutes  to  the  surface.  Orion  has 
three  main  elements — the  crew  module  (capsule), 
service  module/spacecraft  adapter,  and  launch  abort 
system. 


Source:  Lockheed  Martin  Space  Systems  (artist  depiction). 
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(8/09) 
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(3/15) 

Project  Essentials 

NASA  Center  Lead:  Johnson  Space  Center 

International  Partner:  None 

Major  Contractors:  Lockheed  Martin 

Projected  Launch  Date:  March  2015 
Launch  Location:  Kennedy  Space  Center,  Fla. 
Launch  Vehicle:  Ares  I 

Mission  Duration:  Varied  based  on  destination 


Project  Performance 
(then  year  dollars  in  millions) 

Latest 
(Feb.  2011) 

Preliminary  Estimate  of 

Project  Life  Cycle  Cost*  $20,000  to  $29,000 

*  This  estimate  is  preliminary,  as  the  project  is  in  formulation 
and  there  is  still  uncertainty  in  the  value  as  design  options 
are  explored.  NASA  uses  these  estimates  for  planning 
purposes.  This  estimate  is  for  the  Orion  vehicle  only. 


Launch  Schedule  3/2015 


Recent/Continuing  Project 
Challenges 

»-  Funding  Stability 
»-  Technology  Issues 

Previously  Reported  Challenges 

»-  Contractor  Performance 


Project  Summary 

The  President’s  fiscal  year  2011  budget  proposed 
cancellation  of  the  Orion  project  leading  to  uncertainty, 
both  financial  and  programmatic,  within  the  project.  Given 
constrained  resources,  the  project  prioritized  work  and 
did  not  accomplish  some  of  the  work  originally  planned 
for  2010.  The  project  did,  however,  successfully  complete 
a  test  of  the  launch  abort  system  and  continue  progress 
on  mitigating  other  technical  challenges.  In  early  fall  2010 
Congress  passed  the  NASA  Authorization  Act  of  2010 
directing  NASA  to  utilize  existing  Orion  contracts  and 
capabilities  to  the  extent  practicable. 
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Orion  Crew  Exploration  Vehicle 

Project  Update 

The  President  proposed  cancellation  of  the  Constellation  Program,  including  the  Orion  project,  in  his 
fiscal  year  2011  budget  request.  This  proposal  led  to  much  debate  within  Congress  and  uncertainty,  both 
financial  and  programmatic,  within  the  project.  As  a  result,  the  project  prioritized  work  for  the  year  and  did 
not  complete  some  of  the  work  originally  planned  for  2010.  In  early  fall  2010,  Congress  passed  the  NASA 
Authorization  Act  of  2010,  which  directed  NASA  to  continue  development  of  a  multipurpose  crew  vehicle 
capable  of  reaching  near-Earth  and  beyond  near-Earth  orbit  no  later  than  December  2016.  In  developing  this 
vehicle,  Congress  directed  the  agency  to  continue  to  advance  development  of  the  human  safety  features, 
designs,  and  systems  in  the  Orion  project  and  to  utilize  existing  contracts  and  capabilities  to  the  extent 
practicable. 

Funding  Issues:  Funding  shortfalls  and  uncertainty  have  impacted  workforce  availability,  shifted  the 
Orion  schedule  and  testing  strategy,  and  deferred  procurement  of  new  items.  For  example,  during  fiscal 
year  2010,  NASA  and  Lockheed  Martin  had  arranged  an  agreement  under  which  Lockheed  Martin  would 
have  performed  $200  million  worth  of  work  during  the  current  fiscal  year  that  NASA  would  pay  for  during 
later  phases  of  the  Orion  project.  However,  according  to  project  officials,  NASA  decided  not  to  execute  the 
agreement  because  NASA  lacked  sufficient  budget  authority  to  obligate  funds  to  pay  for  the  work.  This  left 
the  project,  and  the  entire  Constellation  Program,  without  the  $200  million  worth  of  work  that  they  had 
expected  and  with  limited  resources  for  completing  the  remaining  work  for  fiscal  year  2010,  so  therefore, 
the  project  prioritized  development  activities  and  tests.  The  Orion  project  received  nearly  $166  million  of 
funding  under  the  American  Recovery  and  Reinvestment  Act  of  2009  that  according  to  project  officials 
halted  layoffs  at  Lockheed  Martin  and  helped  the  project  overcome  technical  challenges.  The  value  of  the 
development  contracts  for  Orion  has  increased  by  $2.5  billion  since  2006. 

Technology  Issues:  The  Orion  project  identified  one  critical  heritage  technology  for  the  spacecraft:  the 
thermal  protection  system,  or  heatshield,  that  is  required  for  the  spacecraft  to  survive  reentry  from  earth 
orbit.  According  to  project  officials,  the  new  material  for  the  heatshield  has  been  tested  against  the  material 
used  in  the  Apollo  program  and  performs  as  well  as  or  better  than  the  heritage  material.  However,  given 
the  current  funding  constraints  and  uncertainty  surrounding  the  Orion  project,  the  Orion  project  office 
prioritized  development  activities,  and  while  the  heatshield  development  and  testing  are  continuing  on  plan, 
the  determination  of  the  manufacturing  processes  has  been  deferred. 

In  addition,  development  of  the  launch  abort  system,  which  would  pull  the  Orion  capsule  away  from  the 
Ares  I  launch  vehicle  in  the  case  of  a  catastrophic  problem  during  launch,  remains  a  high  risk  area  even 
though  it  was  not  identified  as  a  critical  technology.  In  May  2010,  the  project  tested  the  Launch  Abort  System 
in  the  Orion’s  Pad  Abort  (PA-1)  test.  According  to  project  officials,  PA-1  was  an  important  developmental 
milestone  for  the  launch  abort  system,  but  certain  items  that  were  found  during  the  test  will  require  design 
modifications  to  the  system  that  will  not  be  tested  until  funding  is  available.  The  project  has  also  developed 
a  new  controller  for  the  launch  abort  system,  and  planned  to  test  it  in  the  ascent  abort  test  in  2012.  However, 
due  to  funding  instability,  it  is  unknown  when  and  if  this  test  will  take  place. 

Project  Office  Comments 

The  Orion  project  office  provided  technical  comments  on  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  also  commented  that  the  project  has  continued  its  work  on 
the  Constellation  program.  Reductions  in  planned  work  content  were  made  to  ensure  availability  of  funds 
required  to  complete  work  already  under  contract.  These  reductions  have  made  it  difficult  for  NASA  to 
achieve  some  of  its  goals  and  outcomes  planned  for  fiscal  year  2010.  NASA  remains  poised  to  leverage 
Constellation  assets  to  contribute  to  future  exploration  beyond  low-Earth  orbit. 
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Radiation  Belt  Storm  Probes  (RBSP) 


The  Radiation  Belt  Storm  Probes  (RBSP)  mission 
will  explore  the  Sun’s  influence  on  the  Earth  and 
near-Earth  space  by  studying  the  planet’s  radiation 
belts  at  various  scales  of  space  and  time.  This  insight 
into  the  physical  dynamics  of  the  Earth’s  radiation 
belts  will  provide  scientists  data  to  make  predictions 
of  changes  in  this  little  understood  region  of  space. 
Understanding  the  radiation  belt  environment  has 
practical  applications  in  the  areas  of  spacecraft 
system  design,  mission  planning,  spacecraft 
operations,  and  astronaut  safety.  The  two  spacecrafts 
will  measure  the  particles,  magnetic  and  electric 
fields,  and  waves  that  fill  geospace  and  provide  new 
knowledge  on  the  dynamics  and  extremes  of  the 
radiation  belts. 


Source:  ©2010  The  Johns  Hopkins  University/Applied  Physics  Laboratory.  All  Rights  Reserved. 
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Project  Essentials 

Project  Performance 

NASA  Center  Lead:  Goddard  Space  Flight  Center 

(then  year  dollars  in  millions) 

Partner:  National  Reconnaissance  Office 

Baseline  Est. 

Latest 

(FY  2009) 

(Feb.  2011) 

Change 

Major  Contractors:  John  Hopkins  University/Applied 
Physics  Laboratory 

Total  Project  Cost 
Formulation  Cost 

$685.8 

$88.2 

$685.9 

$88.2 

0.0% 

0.0% 

Projected  Launch  Date:  May  18,  2012 

Launch  Location:  Cape  Carnaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Development  Cost 
Operations  Cost 

$533.9 

$63.7 

$534.0 

$63.7 

0.0% 

0.0% 

Mission  Duration:  2  years 

Launch  Schedule 

5/2012 

5/2012 

0  months 

Project  Summary 

RBSP  project  officials  reported  parts  failure  and  contractor 
issues  that  may  result  in  the  delayed  delivery  and  integration 
of  two  key  science  instruments.  Project  officials  expect 
delays  in  the  delivery  of  the  Helium-Oxygen-Proton- 
Electron  instrument  due  to  a  parts  functionality  failure 
and  in  the  delivery  of  necessary  flight  hardware  for  the 
MagEIS  instrument  that  may  impact  its  integration  with  the 
spacecraft.  RBSP’s  systems  integration  review  was  held  in 
October  2010. 


Recent  Project  Challenges 

»-  Parts  Issues 
»-  Contractor  Issues 
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Radiation  Belt  Storm  Probes  (RBSP) 

Project  Update 

Parts  Issues:  RBSP  project  officials  expect  delays  in  the  delivery  and  integration  the  Helium-Oxygen- 
Proton-Electron  (HOPE)  instrument.  Delivery  of  HOPE  may  be  delayed  due  to  a  parts  functionality  failure 
within  the  high  voltage  Optocoupler.  Currently,  the  project  considers  this  parts  issue  a  risk  to  mission 
cost  and  schedule.  However,  the  project  manager  reported  that  there  are  sufficient  schedule  reserves  and 
that  they  have  confidence  that  issues  can  be  resolved  without  schedule  growth.  Project  officials  said  that 
other  NASA  missions  had  issues  with  the  same  part.  The  manufacturer  is  working  to  develop  a  revised 
Optocoupler  to  meet  multiple  mission  needs. 

NASA  provided  instructions  that  prohibited  the  use  of  certain  connectors  as  part  of  their  ongoing  monitoring 
of  quality  parts  and  qualification  standards,  which  caused  the  project  to  review  the  type  of  connectors 
used  in  the  observatory  and  replace  the  connectors  as  applicable.  The  project  has  successfully  qualified  a 
connector  to  replace  the  NASA-prohibited  connectors.  The  new  connector  has  been  successfully  installed 
on  flight  model  boards  across  the  project.  RBSP  project  officials  classify  the  likelihood  of  an  in-flight  failure 
if  the  prohibited  connectors  were  used  as  very  small;  however,  possible  consequences  including  loss  of  the 
spacecraft  or  an  instrument  are  significant. 

Contractor  Issues:  Delivery  of  the  Magnetic  Electron  Ion  Spectrometer  (MagEIS)  instrument  is  expected 
to  be  delayed  due  to  the  time  a  vendor  is  taking  in  providing  needed  flight  hardware  for  the  instrument.  A 
project  official  reported  that  the  vendor  was  contacted  and  encouraged  to  prioritize  its  commitment  to  the 
RBSP  contract.  However,  officials  reported  that  the  project  underwent  a  schedule  replan  to  accommodate 
the  late  delivery  and  integration  of  MagEIS.  This  replan  maintains  the  launch  readiness  date  by  re-ordering 
the  observatory  integration  and  test  flow  and  changing  selected  subsystem  and  instrument  delivery  dates. 

Other  Issues  to  be  Monitored:  Project  officials  indicated  that  one  of  the  primary  challenges  for  RBSP  is 
developing  a  spacecraft  capable  of  withstanding  the  high  levels  of  radiation  that  it  will  encounter  during  the 
mission.  RBSP  includes  many  design  elements,  such  as  aluminum  shielding  around  all  major  subsystems, 
and  is  undergoing  extensive  testing  and  qualification  to  ensure  sufficient  “radiation  hardening.”  The  project 
manager  reported  that  spacecraft  electronic-related  parts  radiation  testing  is  nearly  complete  with  no 
problems  reported. 

Only  69  percent  of  the  engineering  design  drawings,  instead  of  the  planned  87  percent,  were  released  by  the 
December  2009  critical  design  review  (CDR)  for  RBSP.  In  April  2010,  the  project  had  released  93  percent 
of  its  drawings.  Project  officials  said  that  RBSP  was  the  first  project  at  the  Johns  Hopkins  University/ 
Applied  Physics  Laboratory  to  use  a  new  tracking  package  for  reviewing  and  approving  design  drawings  and 
therefore  experienced  some  delays  in  releasing  drawings  at  CDR.  Project  officials  reported  that  there  have 
been  only  minimal  design  changes  since  the  CDR  and  there  are  no  significant  design  changes  expected  in 
the  future. 

Project  Office  Comments 

The  RBSP  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  also  commented  that  the  System  Integration  Review  was 
conducted  on  12-14  October  2010,  with  the  Standing  Review  Board  recommending  that  the  Project  be 
allowed  to  proceed  into  observatory  integration  and  test. 
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Soil  Moisture  Active  and  Passive  (SMAP) 


NASA’s  Soil  Moisture  Active  and  Passive  (SMAP) 
is  one  of  four  first-tier  missions  recommended  by 
the  National  Research  Council’s  2007  Earth  Science 
Decadal  Survey.  SMAP  leverages  previous  Earth 
Science  missions  and  is  based  on  the  soil  moisture 
and  freeze/thaw  mission  concept  developed  by  an 
earlier  mission  known  as  Hydros.  The  SMAP  mission 
will  provide  new  information  on  global  soil  moisture 
and  its  freeze/thaw  state  enabling  new  advances  in 
hydrospheric  science  and  applications.  The  measures 
will  improve  understanding  of  regional  and  global 
water  cycles,  improve  weather  forecasts,  flood  and 
drought  forecasts,  and  predictions  of  agricultural 
productivity  and  climate  changes. 

Source:  Jet  Propulsion  Laboratory  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Jet  Propulsion  Laboratory 

International  Partner:  None 

Major  Contractors:  None 

Projected  Launch  Date:  November  2014 
Launch  Location:  TBD 
Launch  Vehicle:  TBD 

Mission  Duration:  3  years 


Project  Performance 
(then  year  dollars  in  millions) 

Latest 
(Feb.  2011) 

Preliminary  Estimate  of 

Project  Life  Cycle  Cost*  $780  -  $900 

*  This  estimate  is  preliminary,  as  the  project  is  in  formulation 
and  there  is  still  uncertainty  in  the  value  as  design  options 
are  explored.  NASA  uses  these  estimates  for  planning 
purposes. 


Launch  Schedule  11/2014 


Recent  Project  Challenges 

»-  Funding  Issues 
»-  Launch  Issues 


Project  Summary 

SMAP  received  $64  million  in  American  Recovery  and 
Reinvestment  Act  of  2009  funds,  as  well  as  funding  from 
the  President’s  global  climate  initiative,  that  the  project 
used  to  address  key  mission  and  implementation  risks 
during  formulation  and  to  accelerate  the  launch  readiness 
date  from  May  2015  to  November  2014.  The  project  is 
currently  being  designed  to  multiple  launch  vehicle 
specifications  and  is  tracking  the  timing  of  the  launch 
vehicle  selection  as  a  top  risk. 
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Soil  Moisture  Active  and  Passive  (SMAP) 

Project  Update 

Funding  Issues:  SMAP  entered  formulation  in  September  2008  and  the  Jet  Propulsion  Lab  (JPL)  was 
selected  as  the  lead  implementation  center  in  January  2009.  NASA  officials  stated  that  SMAP  was  budgeted 
$30  million  in  funding  from  the  President’s  global  climate  initiative  and  $64  million  in  funding  from  the 
American  Recovery  and  Reinvestment  Act  of  2009,  which  the  project  used  to  accelerate  the  launch  date 
from  May  2015  to  November  2014. 

Launch  Issues:  Late  launch  vehicle  selection  is  one  of  the  top  risks  the  project  is  monitoring.  SMAP  is 
currently  being  designed  to  fit  the  specifications  for  three  launch  vehicles,  including  exploring  a  partnership 
for  a  DOD-provided  launch  service  on  the  Minotaur  IV.  While  designing  to  accommodate  multiple  launch 
vehicles  is  possible,  a  project  official  said  that  it  limits  design  capabilities  and  can  raise  costs  to  the  program 
as  a  result.  Project  officials  stated  that  no  certified  medium  capability  vehicle  is  currently  available.  The 
Falcon  9,  which  is  available  under  the  current  Launch  Services  contract,  has  yet  to  be  certified,  and  if 
selected,  the  mission  launch  date  will  be  tied  to  a  successful  certification  of  the  launch  vehicle.  NASA  is 
preparing  a  solicitation  to  acquire  launch  services  and,  if  commercial  vehicles  are  not  reasonably  available, 
it  may  request  approval  by  the  Secretary  of  Defense  and  submit  a  certification  to  Congress  for  authorization 
to  partner  with  DOD  to  use  the  Minotaur  IV.  The  current  timeline  for  launch  vehicle  selection  may  result  in  a 
decision  after  the  project’s  preliminary  design  review  (PDR). 

Other  Issues  to  be  Monitored:  Project  officials  stated  that  an  early  focus  on  risk  management  enabled 
SMAP  to  mitigate  several  top  mission  and  implementation  risks  related  to  the  aggressive  schedule  and  the 
scientific  outputs  of  the  mission.  For  example,  the  project  developed  an  end-to-end  science  measurement 
simulation  to  increase  the  data  volume  requirements.  The  project  expects  to  mitigate  several  other 
development  risks  by  the  mission  PDR  in  March  2011.  For  example,  the  project  reported  it  has  three 
heritage  technologies — the  radar,  radiometer,  and  the  reflector  boom  assembly — all  of  which  it  will  adapt 
for  application.  None  of  these  technologies,  however,  is  currently  mature.  The  project  is  tracking  the 
radiometer  as  a  project  risk  since  it  requires  additional  Spectral  Filtering  for  Radio  Frequency  Interference 
(RFI)  mitigation.  The  project  has  identified  the  spectral  filtering  as  a  critical  technology.  Due  to  its  extensive 
heritage,  the  project  is  accepting  the  potential  risk  in  cost  growth  and  the  technical  risks  with  a  verification 
and  validation  (V&V)  program  that  includes  a  comprehensive  set  of  assembly  and  system  level  analyses. 
There  is  a  cost  risk,  however,  associated  with  the  V&V  program  if  the  project  determines  that  additional 
tests  and  analyses  are  required. 

SMAP  leverages  other  Earth  Science  projects,  namely  the  Aquarius  project,  which  is  in  the  implementation 
phase,  and  the  Hydros  project  that  was  discontinued  in  2005  due  to  lack  of  available  funding.  Although 
SMAP  has  no  funding  partners,  the  National  Oceanic  and  Atmospheric  Administration,  the  U.S.  Department 
of  Agriculture,  and  DOD  are  all  actively  engaged  with  SMAP  to  develop  an  applications  plan  for  the  data. 

Project  Office  Comments 

The  SMAP  project  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated  as 
appropriate.  The  project  officials  also  commented  that  the  target  launch  readiness  date  of  November  2014 
is  a  planning  date  at  this  point  and  can  change  as  funding,  scope  and  schedule  are  brought  into  mutual 
alignment.  NASA  will  not  formally  commit  to  a  launch  readiness  date  until  Project  Confirmation,  Key 
Decision  Point  C,  currently  scheduled  for  summer  2011. 
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Solar  Probe  Plus  (SPP) 

Solar  Probe  Plus  (SPP)  will  explore  the  Sun's  outer 
atmosphere,  or  corona,  as  it  extends  into  space. 

The  spacecraft  will  orbit  the  Sun  24  times  and  its 
instruments  will  observe  the  generation  and  flow  of 
solar  wind  from  very  close  range.  By  observing  the 
corona,  where  solar  energetic  particles  are  energized, 
there  is  potential  to  further  science  in  terms  of 
shedding  light  on  two  central  issues  of  heliophysics: 
the  origin  and  evolution  of  solar  wind,  and  why 
the  sun’s  outer  atmosphere  is  so  much  hotter  than 
the  visible  surface.  In  order  to  achieve  its  mission, 
parts  of  the  spacecraft  must  be  able  to  withstand 
temperatures  exceeding  2,500  degrees  Fahrenheit,  as 
well  as  endure  blasts  of  extreme  radiation. 

Source:  ©2010  Johns  Hopkins  University/Applied  Physics  Laboratory  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 

International  Partner:  None 

Major  Contractors:  Johns  Hopkins  University  Applied 
Physics  Laboratory,  Southwest  Research  Institute,  Naval 
Research  Laboratory,  University  of  California  at  Berkeley 
Space  Sciences  Laboratory,  Smithsonian  Astrophysical 
Observatory. 

Projected  Launch  Date:  2018 
Launch  Location:  Eastern  Range 
Launch  Vehicle:  TBD 

Mission  Duration:  7  years 


Project  Performance 
(then  year  dollars  in  millions) 

Latest 
(Feb.  2011) 

Preliminary  Estimate  of 

Project  Life  Cycle  Cost*  not  available 

*The  project  has  not  yet  reached  the  point  in  the  acquisition 
life  cycle  where  a  preliminary  life  cycle  cost  estimate  would 
normally  be  developed. 


Launch  Schedule  8/2018 


Recent  Project  Challenges 

»-  Launch  Issues 


Project  Summary 

SPP  is  early  in  formulation,  and  therefore  is  unable 
to  provide  official  cost  and  schedule  data  at  this  time. 
Currently,  the  probe  will  fly  within  closer  proximity  to  the 
Sun  than  any  other  spacecraft.  Chief  risks  to  the  project 
in  terms  of  cost  and  schedule  include  development  of  a 
sunshield  capable  of  protecting  the  instruments  from  the 
harsh  near-Sun  environment,  development  of  a  cooling 
system  for  the  retractable  solar  array  panels,  and 
achieving  the  total  launch  energy  to  get  the  spacecraft  to 
its  long-range  destination. 
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Common  Name:  SPP 


Solar  Probe  Plus  (SPP) 

Project  Update 

Launch  Issues:  SPP  project  officials  reported  that  one  of  the  mission’s  key  challenges  is  achieving  the  total 
launch  energy  necessary  to  launch  the  spacecraft  toward  its  long  range  destination.  The  mission  will  most 
likely  require  the  use  of  an  upper  stage  solid  rocket  propellant  to  provide  sufficient  launch  energy  to  set  the 
spacecraft  on  a  trajectory  to  achieve  solar  exploration.  Project  officials  reported  that  they  are  working  to 
understand  the  performance  of  the  standard  stage  and  possible  enhancements  to  upper  stage  performance 
should  this  be  needed.  These  enhancements  could  include  the  possible  use  of  a  higher  energy  propellant 
and  a  composite  case  for  mass  efficiency.  The  project  commissioned  a  trade  study  which  seeks  to  identify 
the  optimal  combination  of  launch  vehicle  and  propellant  upper  stage  to  use  for  the  launch.  Project  officials 
anticipate  the  study  to  be  completed  by  the  Mission  Design  Review,  currently  scheduled  for  May  2011. 

Other  Issues  to  be  Monitored:  A  key  challenge  of  the  SPP  mission  will  be  the  development  of  critical 
technologies  allowing  science  instruments  to  function  within  the  harsh  near-Sun  environment.  Although  still 
in  the  concept  and  technology  development  phase,  project  officials  reported  that  the  Thermal  Protection 
System  (TPS) — a  carbon-foam  filled  sun  shield  that  will  measure  over  8  feet  in  diameter — would  sit  atop 
the  spacecraft  shielding  instruments  from  the  direct  heat  and  radiation  of  the  Sun.  Project  officials  reported 
that  they  have  already  completed  production  of  a  30-inch  square  prototype  TPS  shield,  but  at  this  time  the 
technology  is  not  fully  mature.  A  full  prototype  of  this  technology  is  expected  to  be  matured  and  built  during 
Phase  B. 

A  second  area  of  mission  technology  development  concerns  the  production  of  two  sets  of  solar  arrays — 
essentially  solar  power  generators — that  will  retract  and  extend  as  the  spacecraft  moves  toward  or 
away  from  the  Sun.  A  solar  array  cooling  system  will  be  used  to  ensure  the  solar  panels  stay  at  required 
temperatures.  Project  officials  reported  that  the  cooling  system  will  need  the  capacity  to  dissipate  up  to 
5,000  watts  of  thermal  energy  during  the  spacecraft’s  closest  approach  to  the  Sun.  In  order  to  mitigate 
mission  risk,  a  back-up  pump  for  the  cooling  system  is  planned  to  be  integrated  should  the  first  pump  fail. 
However,  as  is  the  case  with  the  TPS,  it  will  be  impossible  to  replicate  the  extreme  conditions  the  probe  will 
be  exposed  to  during  its  closest  proximity  to  the  Sun. 

Although  the  key  technologies  will  be  tested  in  representative  environments  it  will  be  impossible  to  replicate 
the  extreme  conditions  the  fully  assembled  probe  will  be  exposed  to  during  its  closest  proximity  to  the 
Sun  requiring  simulators  for  the  TPS  and  Solar  Arrays  in  systems  test.  Thus,  the  functionality  of  the  entire 
spacecraft  in  the  near-Sun  environment  cannot  be  verified  fully  through  testing  prior  to  launch. 

An  Announcement  of  Opportunity  was  issued  in  December  2009  and  project  officials  reported  that  thirteen 
science  proposals  were  considered  by  a  panel  of  NASA  and  other  scientists.  In  2010,  the  project  selected 
five  science  investigations,  which  when  awarded  will  have  a  combined  value  of  approximately  $165  million 
for  preliminaiy  analysis,  design,  development,  and  testing. 

Project  Office  Comments 

The  SPP  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were  incorporated 
as  appropriate.  Project  officials  also  commented  that  SPP  is  making  progress  going  through  formulation. 


Page  74 


GAO-1 1-239SP  Assessments  of  Selected  Large-Scale  Projects 


Common  Name:  SOFIA 


Stratospheric  Observatory  for  Infrared  Astronomy  (SOFIA) 

SOFIA  is  a  joint  project  between  NASA  and  the 
German  Space  Agency  to  install  a  2.5  meter  telescope 
in  a  specially  modified  Boeing  747SP  aircraft.  This 
airborne  observatory  is  designed  to  provide  routine 
access  to  the  visual,  infrared,  far-infrared,  and 
sub-millimeter  parts  of  the  spectrum.  Its  mission 
objectives  include  studying  many  different  kinds 
of  astronomical  objects  and  phenomena,  including 
star  birth  and  death;  the  formation  of  new  solar 
systems;  planets,  comets,  and  asteroids  in  our  solar 
system;  and  black  holes  at  the  center  of  galaxies. 

Interchangeable  instruments  for  the  observatory 
are  being  developed  to  allow  a  range  of  scientific 
measurement  to  be  taken  by  SOFIA. 

Source:  SOFIA  First  Light  Image  Composite. 
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Project  Essentials 

NASA  Center  Lead:  Dryden  Flight  Research  Center 
International  Partner:  German  Space  Agency 

Major  Contractors:  L3  Communications,  WMPC  Products 
Corporation,  University  Space  Research  Association 

Projected  Operational  Capability:  December  201 4 
Aircraft:  Modified  747SP 

Sortie  Location:  Dryden  Flight  Research  Center,  Calif. 
Mission  Duration:  20  years  of  science  mission  flights 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 
(FY  2007) 

Total  Project  Cost  $2954.5 
Formulation  Cost  $35.0 
Development  Cost  $919.5 
Operations  Cost  $2000.0 


Latest 

(Feb.  2011) 

Change 

$3002.9 

1 .6% 

$35.0 

0.0% 

$1128.4 

22.7% 

$1839.5 

-8.0% 

Operating  Schedule  12/2013  12/2014  12  months 


Recent/Continuing  Project 
Challenges 

»-  Technology  Issues 
»-  Design  Issues 
»-  Contractor  Issues 

Previously  Reported  Challenges 

»-  Funding  Issues 


Project  Summary 

Since  our  last  review,  SOFIA  has  experienced  a  delay  in 
the  delivery  of  hardware  from  vendors  and  development 
issues  surrounding  the  Cavity  Door  Drive  System.  While 
this  resulted  in  a  7-month  slip  to  initiation  of  science 
flights  in  December  2010,  the  program  completed  a 
significant  progress  milestone  with  the  completion  of  the 
first  light  flight  on  May  25,  2010.  In  2009  and  2010,  NASA 
reported  to  the  Congress  that  SOFIA  exceeded  both  its 
cost  and  schedule  baselines. 
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Common  Name:  SOFIA 


Stratospheric  Observatory  for  Infrared  Astronomy  (SOFIA) 

Project  Update 

As  required  by  law,  NASA  reported  to  the  Congress  in  2009  and  2010  that  SOFIA  exceeded  its  2007 
development  cost  baseline  by  more  than  15  percent  and  its  schedule  baseline  by  more  than  6  months. 
SOFLA’s  development  costs  have  increased  more  than  268  percent,  over  $1.1  billion,  since  its  1995  estimate. 
These  cost  increases  are  partly  due  to  challenges  with  modification  of  the  aircraft  to  be  used  for  SOFIA  and 
more  recently  development  of  the  Cavity  Door  Drive  System  (CDDS).  This  year,  project  officials  told  us 
SOFLA’s  development  costs  increased  due  to  increased  flight  hanger  costs.  Some  data  for  the  project  was  not 
provided  by  NASA  because,  according  to  project  officials,  the  project  documentation  did  not  transfer  in  its 
entirety  from  Ames  Research  Center  to  Dryden  Flight  Research  Center. 

Technology  Issues:  We  could  not  assess  the  technology  maturity  of  the  overall  project  as  NASA  did  not 
provide  information  for  heritage  technologies  related  to  the  aircraft  modification.  Data  provided  for 
development  of  the  instruments  that  will  fly  on  SOFIA  generally  indicates  a  high  level  of  technology 
maturity.  Many  of  these  technologies  have  already  been  used  on  ground-based  telescopes.  Project  officials 
told  us  that  of  the  eight  first  generation  science  instruments,  one  instrument  was  flown  on  the  first  light 
flight  in  May  2010,  one  instrument  has  been  installed  and  tested  on  the  ground,  one  instrument  is  awaiting 
installation,  and  four  instruments  will  be  installed  by  2013. 

Design  Issues:  We  were  unable  to  determine  design  stability  of  the  instruments  since  the  drawings  were  still 
preliminaiy  at  the  critical  design  review.  Last  year,  project  officials  reported  that  design  work  on  SOFIA  was 
97  percent  complete  and  that  all  designs  would  be  complete  by  2011.  However,  due  to  problems  with  the 
CDDS  vendor  and  longer-than-anticipated  door  testing,  initial  science  flights  have  been  delayed  one  year. 
Because  modifications  to  several  subsystems  will  be  ongoing  during  the  early  science  missions,  project 
officials  told  us  designs  will  not  be  finalized  until  2014  when  the  project  is  scheduled  to  begin  operations.  A 
date  for  the  preliminary  design  review  was  not  provided  by  NASA. 

Contractor  Issues:  Since  our  last  review,  the  SOFIA  project  has  experienced  at  least  a  6-month  slip  in 
the  scheduled  commencement  of  initial  science  flights  due  to  late  delivery  of  hardware  and  software  in 
the  CDDS  and  rework  of  vendor  supplied  hardware.  The  project  found  problems  with  software  quality 
assurance,  which  indicated  later  on  that  there  were  problems  with  hardware  quality  assurance  and  required 
a  rebuild  of  the  CDDS  components.  NASA  consequently  reduced  the  contractor’s  management  role  for 
both  development  and  operations  of  SOFIA  and  utilized  government  personnel  to  perform  these  functions 
in  house  and  to  complete  the  CDDS.  The  project  successfully  completed  the  first  open  door  flight  test  on 
December  18,  2009,  and  experienced  no  anomalies.  To  date,  the  project  has  conducted  three  open  door 
landings,  two  of  which  were  unplanned  and  caused  by  nuisance  faults.  The  project  manager  stated  that  in 
the  open  door  testing  process  there  was  a  high  probability  of  a  halt  in  the  door  system  and  the  project  was 
prepared  for  this  occurrence.  He  stated  that  there  is  no  backup  door  opening  system,  but  that  the  project 
did  have  a  default  reset  for  door  issues  in  flight.  The  project  continues  to  troubleshoot  development  of  the 
CDDS  and  is  utilizing  an  independent  consultant  to  investigate  the  system  and  recommend  future  upgrades. 
In  August  2010,  the  project  completed  its  second  segment  of  flight  tests  with  its  telescope  door  open  to 
prepare  the  observatory  for  early  science  missions. 

Project  Office  Comments 

The  SOFIA  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  also  commented  that  the  SOFIA  project  has  made  progress 
toward  the  initiation  of  science  observations. 
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Common  Name:  TDRS 


Tracking  and  Data  Relay  Satellite  (TDRS)  Replenishment 


The  Tracking  and  Data  Relay  Satellite  (TDRS) 

System  consists  of  in-orbit  communication  satellites 
stationed  at  geosynchronous  altitude  coupled  with 
two  ground  stations  located  in  New  Mexico  and 
Guam.  The  satellite  network  and  ground  stations 
provide  mission  services  for  near-Earth  user  satellites 
and  orbiting  vehicles.  TDRS  K  and  L  are  the  11th  and 
12th  satellites,  respectively,  to  be  built  for  the  TDRS 
system  and  will  contribute  to  the  existing  network 
by  providing  high  bandwidth  digital  voice,  video,  and 
mission  payload  data,  as  well  as  health  and  safety 
data  relay  services  to  Earth-orbiting  spacecraft,  such 
as  the  International  Space  Station. 


Source:©  Boeing  (artist  depiction). 
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Project  Essentials 

NASA  Center  Lead:  Goddard  Space  Flight  Center 
Partner:  Non-NASA  Agencies 

Major  Contractors:  Boeing  Satellite  Systems,  United 
Launch  Alliance  (ULA) 

Projected  Launch  Date:  TDRS  K  -  December  2012 
TDRS  L  -  December  2013 
Launch  Location:  Cape  Canaveral  AFS,  Fla. 

Launch  Vehicle:  Atlas  V 

Mission  Duration:  15  years 


Recent  Project  Challenges 

»-  Parts  Issues 


Project  Performance 
(then  year  dollars  in  millions) 


Baseline  Est. 

Latest 

(FY  2010) 

(Feb.  2011) 

Change 

Total  Project  Cost* 

$451.3 

$434.1 

-3.8% 

Formulation  Cost 

$241.9 

$241 .9 

0.0% 

Development  Cost 

$209.4 

$192.2 

-8.2% 

Operations  Cost 

$0.0 

$0.0 

0.0% 

Launch  Schedule 

K  12/2012 

L  12/2013 

K  12/2012 

L  12/2013 

0  months 
0  months 

*  Represents  the  estimate  of  NASA  funding  and  does  not 
include  expected  partner  contributions. 


Project  Summary 

The  TDRS  project  identified  an  issue  with  contamination 
of  the  lubricants  in  the  reaction  wheel  assemblies. 

The  cost  impact  of  this  issue  is  borne  by  the  prime 
contractor.  In  June  2010,  the  project  awarded  a  contract 
to  enhance  existing  ground-system  architecture  to  ensure 
the  TDRS  system  continues  providing  space-to-ground 
telecommunications.  However,  even  with  the  successful 
launch  of  TDRS  K  and  L,  NASA  is  only  able  to  guarantee 
continuity  of  service  of  the  TDRS  system  through  fiscal 
year  2016. 
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Common  Name:  TDRS 


Tracking  and  Data  Relay  Satellite  (TDRS)  Replenishment 

Project  Update 

Parts  Issues:  In  2010,  TDRS  project  officials  discovered  that  the  lubricant  in  the  reaction  wheel  assemblies 
was  contaminated  by  silicone.  The  project  initially  reported  that  it  may  take  up  to  18  months  for  the  original 
supplier  to  provide  replacements  and  that  no  other  appropriate  reaction  wheels  are  in  production  by 
alternative  vendors.  However,  project  officials  expected  that  replacement  reaction  wheels  would  be  made 
available  in  November  and  December  2010,  which  equates  to  an  approximate  2-month  delay  to  scheduled 
wheel  delivery  dates. 

Other  Issues  to  be  Monitored:  In  June  2010,  a  cost-plus-award  fee  contract  was  awarded  to  modernize 
the  ground  based  communication  systems  needed  for  TDRS  K  and  L.  In  order  to  maximize  the  capabilities 
of  TDRS  K,  necessary  enhancements  to  the  ground  system  must  be  prioritized  within  the  2  years  prior  to 
launch  in  2012.  TDRS  K  and  L  are  being  designed  with  high-bandwidth  communication  abilities  including 
the  transmission  of  images,  video,  voice,  and  other  digital  data  from  Earth-orbiting  spacecraft  to  the  ground. 
The  ground-based  beamforming  architecture  at  the  White  Sands  Complex  in  New  Mexico  is  currently  being 
modified  to  provide  TDRS  K  and  L  compatible  beamformers  for  the  ground  station.  Project  officials  reported 
that  the  switch  to  ground-based  beamforming  was  required  to  provide  compatibility  with  network  demand 
services  developed  in  the  late  1990’s.  Project  officials  recognize  challenges  with  updating  ground  segment 
equipment  describing  some  current  instruments  as  vintage  early  1990’s  and  facing  obsolescence  issues. 

The  TDRS  System  is  considered  by  NASA  to  be  a  basic  agency  capability  and  a  national  resource.  The  Space 
Shuttle  and  many  near-Earth  spacecraft  are  totally  dependent  upon  the  satellite  system  for  communication, 
and  therefore,  NASA  considers  the  TDRS  Replenishment  project  critical  in  terms  of  achieving  launch 
schedule.  However,  even  with  the  successful  launch  of  TDRS  K  and  L,  continuity  of  service  for  TDRS 
System  can  only  be  ensured  for  NASA  and  other  government  agency  users  through  approximately  fiscal 
year  2016  at  current  support  levels.  The  primary  reason  for  this  is  due  to  an  aging  fleet  of  satellites.  The 
first  TDRS  satellite,  now  decommissioned,  has  been  in  Earth  orbit  since  1983.  According  to  a  project 
official,  the  current  fixed  price  development  contract  for  TDRS  K  and  L  includes  an  option  to  produce  two 
additional  TDRS  satellites — designated  M  and  N — and  the  addition  of  these  two  satellites  could  extend 
TDRS  system  service  continuity.  However,  in  order  to  exercise  the  options  for  TDRS  M  and  N,  NASA  would 
need  a  financial  commitment  of  $1.2  billion  from  partnership  organizations.  Project  officials  reported  that 
a  decision  on  exercising  the  option  for  TDRS  M  needs  to  be  made  no  later  than  November  30,  2011,  and  no 
later  than  November  30,  2012,  for  TDRS  N. 

Project  Office  Comments 

The  TDRS  project  office  provided  technical  comments  to  a  draft  of  this  assessment,  which  were 
incorporated  as  appropriate.  Project  officials  commented  that  they  agreed  with  the  assessment  as  written. 
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Agency  Comments 
and  Our  Evaluation 


We  provided  a  draft  of  this  report  to  NASA  for  review  and  comment.  In  its 
written  response,  NASA  agreed  with  our  findings  and  stated  that  it  will 
continue  to  identify  and  address  the  challenges  that  may  lead  to  cost  and 
schedule  growth  in  its  projects.  NASA  agreed  that  GAO’s  cost  and  schedule 
growth  figures  reflect  what  the  agency  has  experienced  since  baselines 
were  established  in  response  to  the  2005  statutory  reporting  requirement. 
NASA  also  stated  that  the  average  cost  growth  remains  below  the  15 
percent  threshold  that  requires  Congressional  notification.  While  this  is 
correct,  it  should  be  noted  that  the  notification  requirements  are  for 
individual  projects,  not  the  portfolio  as  a  whole.  In  addition,  NASA 
acknowledges  that  the  current  estimates  for  the  James  Webb  Space 
Telescope  do  not  represent  the  cost  and  schedule  required  to  complete  the 
project,  and  that  the  agency  is  undertaking  a  comprehensive  replanning 
activity  to  establish  the  best  budget  phasing  and  schedule  to  minimize  risk 
and  life-cycle  cost  within  the  overall  constraints  of  its  budget.  We  encourage 
NASA  to  provide  a  revised  budget  and  schedule  for  JWST  that  is  based  on  a 
sound,  knowledge-based  business  case  to  allow  the  project  to  succeed. 


NASA  noted  that  its  projects  are  high-risk,  one-of-a-kind  development 
efforts  that  do  not  lend  themselves  to  all  the  practices  of  a  “business  case” 
approach  that  we  outlined  since  essential  attributes  of  NASA’s  project 
development  differ  from  those  of  a  production  entity.  We  agree  and  do  not 
assess  NASA’s  projects  for  production  maturity.  We  do,  however,  assess 
NASA  projects  at  critical  points  in  the  product  development  process  to 
ensure  that  these  projects  are  proceeding  with  system  development  with 
a  sound  business  case.  At  these  key  junctures  we  have  found  that  NASA 
could  benefit  from  a  more  disciplined  approach  to  its  acquisitions  whereby 
decisions  are  based  upon  high  levels  of  knowledge.  As  we  reported, 
inherent  risks  are  being  heightened  due  to  projects  moving  forward  with 
immature  technologies,  unstable  designs,  and  other  challenges,  leading  to 
cost  and  schedule  increases  that  make  it  hard  for  the  agency  to  manage  its 
portfolio  and  make  informed  investment  decisions.  GAO  looks  forward  to 
working  with  NASA  as  it  develops  metrics  to  better  measure  design  stability 
and  continues  to  refine  the  information  it  uses  to  understand  a  project’s 
status  and  make  informed  decisions. 


NASA  stated  that  the  drawing  release  metric  we  use  to  assess  design 
stability  was  developed  prior  to  the  use  of  computerized  drawings  and  does 
not  take  into  account  improvements  due  to  the  use  of  this  technology.  We 
acknowledge  this  point,  but  our  analysis  of  NASA  projects  shows  that  those 
projects  that  have  met  or  come  close  to  meeting  the  best  practices  drawing 
release  metric  have  fared  better  with  regard  to  cost  and  schedule  than 
those  projects  that  did  not  come  close  to  meeting  the  metric.  Furthermore, 
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in  no  way  does  GAO  portend  that  the  drawing  release  metric  is  the  only 
way  to  assess  design  stability.  Until  NASA  has  taken  steps  to  identify  a 
consistent  and  proven  metric  by  which  to  measure  projects  with  a  portfolio 
perspective,  however,  we  will  continue  to  use  this  metric  to  assess  stability. 
NASA  has  indicated  that  it  will  develop  such  metrics  and  provide  them  to 
GAO  in  March  2011.  We  are  encouraged  by  this  progress  and  look  forward 
to  receiving  the  information. 

NASA  expressed  concern  that  technical  corrections  it  provided  to  our 
2-page  summaries  were  not  fully  accepted.  We  incorporated  the  technical 
comments  where  supporting  documentation  that  meets  our  standards  of 
evidence  was  provided.  We  did  not  incorporate  the  comments  where  this 
information  was  not  provided  or  where  the  change  was  less  a  technical 
correction  and  more  a  difference  of  opinion  between  GAO  and  NASA  based 
on  facts  or  where  space  limitations  required  a  briefer  description  of  an 
issue  than  requested  by  NASA.  As  this  work  will  be  continuing  in  future 
years,  we  will  continue  to  capture  the  progress  made  by  all  the  projects  in 
our  review.  Finally,  we  take  great  strides  to  provide  the  latest  information 
possible  in  our  report.  We  will  continue  to  work  with  NASA  to  ensure  that 
updated  information  is  provided  to  GAO  in  a  timely  manner  so  that  it  can  be 
included  in  our  analysis. 

NASA’s  written  comments  are  reprinted  in  appendix  I.  NASA  also 
provided  technical  comments,  which  we  addressed  throughout  the  report 
as  appropriate  and  where  sufficient  evidence  was  provided  to  support 
significant  changes. 


We  will  send  copies  of  the  report  to  NASA’s  Administrator  and  interested 
congressional  committees.  We  will  also  make  copies  available  to  others 
upon  request.  In  addition,  the  report  will  be  available  at  no  charge  on  GAO’s 
Web  site  at  http://www.gao.gov. 

Should  you  or  your  staff  have  any  questions  on  matters  discussed  in  this 
report,  please  contact  me  at  (202)  512-4841  or  chaplainc@gao.gov.  Contact 
points  for  our  Offices  of  Congressional  Relations  and  Public  Affairs 
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may  be  found  on  the  last  page  of  this  report.  GAO  staff  who  made  major 
contributions  to  this  report  are  listed  in  appendix  IV. 


Cristina  Chaplain 
Director 

Acquisition  and  Sourcing  Management 
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List  of  Congressional  Committees 

The  Honorable  Barbara  A.  Mikulski 
Chairwoman 

The  Honorable  Kay  Bailey  Hutchison 
Ranking  Member 

Subcommittee  on  Commerce,  Justice,  Science,  and  Related  Agencies 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Bill  Nelson 
Chairman 

The  Honorable  John  Boozman 
Ranking  Member 

Subcommittee  on  Science  and  Space 

Committee  on  Commerce,  Science,  and  Transportation 

United  States  Senate 

The  Honorable  Frank  R.  Wolf 
Chairman 

The  Honorable  Chaka  Fattah 
Ranking  Member 

Subcommittee  on  Commerce,  Justice,  Science,  and  Related  Agencies 
Committee  on  Appropriations 
House  of  Representatives 

The  Honorable  Stephen  Palazzo 
Chairman 

The  Honorable  Gabrielle  Giffords 
Ranking  Member 

Subcommittee  on  Space  and  Aeronautics 
Committee  on  Science,  Space,  and  Technology 
House  of  Representatives 
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National  Aeronautics  and  Space  Administration 

Office  of  the  Administrator 

Washington,  DC  20546-0001 

February  23,  201 1 


Ms.  Christina  Chaplain 
Director 

Acquisition  and  Sourcing  Management 
United  States  Government  Accountability  Office 
Washington,  DC  20548 

Dear  Ms.  Chaplain: 

The  National  Aeronautics  and  Space  Administration  (NASA)  appreciates  the 
opportunity  to  comment  on  the  Government  Accountability  Office  (GAO)  draft  report 
entitled  "Assessments  of  Selected  Large-Scale  Projects”  (GAO-1 1-239SP).  NASA 
values  the  continued  open  and  constructive  communications  between  NASA  and  the 
GAO  team  on  this  effort.  NASA  remains  dedicated  to  continuous  improvement  of  its 
acquisition  management  processes  and  performance  and  will  continue  to  work  with  the 
GAO  to  identify  and  address  the  challenges  that  may  lead  to  cost  and  schedule  growth  of 
our  projects. 

We  are  pleased  that  GAO  has  again  recognized  NASA’s  ongoing  efforts  to 
mitigate  acquisition  management  risk  and  lay  a  stronger  foundation  for  reducing  project 
cost  and  schedule  growth.  As  was  highlighted,  NASA  instituted  a  Joint  Cost  and 
Schedule  Confidence  Level  (JCL)  policy  in  2009  to  increase  the  likelihood  of  project 
success  at  the  specified  funding  level.  As  expected  in  2010,  execution  of  the  JCL  process 
prior  to  confirmation  of  several  projects,  including  the  Lunar  Atmosphere  and  Dust 
Environment  Explorer,  the  Mars  Atmosphere  and  Volatile  Evolution  Mission,  and  the 
Orbiting  Carbon  Observatory  2,  increased  insight  by  project  managers,  the  Standing 
Review  Board,  and  NASA  management  surfacing  uncertainties  and  contingencies  with 
the  integrated  cost  and  schedule  plan.  NASA  will  continue  to  assess  the  impact  of 
utilizing  JCLs  on  project  cost  and  schedule  growth  as  these  projects  complete  their 
Systems  Integration  Reviews  in  the  next  two  years.  Furthermore,  with  the  completion 
and  launch  in  201 1  of  three  missions  baselined  under  the  earlier  cost  confidence  level 
policy,  NASA  will  have  a  better  measure  of  the  impact  of  our  acquisition  management 
improvement  efforts  over  the  last  five  years. 

In  its  draft  report,  GAO  states  that  NASA’s  project  development  costs  for  the  16 
projects  in  implementation  in  this  review  have  increased  by  an  average  of  14.6  percent 
from  their  baseline  cost  estimates  and  experienced  an  average  delay  of  eight  months,  an 
improvement  of  three  months  since  the  previous  report.  NASA  agrees  with  the  cost  and 
schedule  growth  figures  that  are  quoted  and  notes  that  the  average  cost  growth  remains 
below  the  15  percent  threshold  which  requires  special  notification  to  Congress. 
Furthermore,  fewer  projects  have  exceeded  this  threshold  since  NASA’s  new  cost 
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estimating  policies  were  put  into  place.  These  figures  are  reflective  of  what  has  been 
experienced  since  baselines  were  established  in  response  to  the  2005  statutory  reporting 
requirement. 

GAO  notes  that  the  calculation  of  NASA’s  average  development  cost  and 
schedule  growth  does  not  include  anticipated  growth  on  the  James  Webb  Space 
Telescope  (JWST)  project.  The  cost  and  schedule  quoted  in  the  draft  GAO  report  are  the 
results  of  a  rough  estimate  by  the  Independent  Comprehensive  Review  Panel  (ICRP) 
which  recently  assessed  JWST  and  do  not  represent  NASA’s  estimate  of  the  cost  and 
schedule  required  to  complete  the  project.  In  response  to  ICRP's  findings  and 
recommendations,  NASA  is  currently  undertaking  a  comprehensive  replanning  activity  to 
establish  the  best  budget  phasing  and  schedule  to  minimize  the  risk  and  life-cycle  cost  of 
JWST  within  the  overall  constraints  of  NASA’s  budget.  The  revised  budget  and  schedule 
will  be  completed  after  the  release  of  the  President's  2012  Budget,  Decisions  resulting 
from  this  replanning  activity,  with  any  required  Agency  offsets,  will  be  reflected  in  the 
President’s  2013  Budget. 

While  NASA  practices  many  elements  of  GAO’s  stated  “business  case"  approach, 
some  essential  aspects  of  NASA's  project  development  differ  from  those  of  a  production 
entity,  which  is  the  basis  for  the  GAO  approach.  NASA’s  projects  are  generally  high- 
risk,  one-of-a-kind  developments  and,  therefore,  do  not  have  a  production  phase.  The 
draft  GAO  report  acknowledges  the  unique  nature  of  NASA  projects  but  applies  a  best 
practices  approach  that  requires  an  incremental  development  process.  NASA's  work 
pushes  the  boundary  of  our  achievements  and  often  requires  leaps,  not  steps,  to 
accomplish  the  mission.  NASA  aims  to  continue  to  innovate  in  an  affordable  and 
sustainable  way  and  will  continue  to  work  with  GAO  to  determine  which  elements  of  the 
approach  arc  valuable  for  informing  improvements  and  which  may  need  to  be  modified 
to  account  best  for  the  complexity  that  surrounds  our  challenging  missions. 

An  area  where  NASA  and  GAO  can  work  together  to  adapt  the  assessment 
approach  is  in  the  determination  of  design  stability.  Although  the  best  practice 
recommends  90  percent  drawing  release  by  Critical  Design  Review  (CDR),  the  drawing 
release  metric  is  a  standard  developed  prior  to  the  use  of  computerized  drawings  and, 
hence,  does  not  take  into  account  improvements  due  to  the  use  of  this  technology.  NASA 
continues  to  develop  metrics  to  measure  design  stability  as  well  as  other  knowledge 
required  to  understand  a  project’s  progress  and  maturity. 

The  draft  GAO  report  notes  challenges  with  launch  vehicles,  specifically, 

NASA’s  transition  plans  for  future  medium-class  launch  vehicles  and  cites  a  previous 
recommendation  that  NASA  perform  detailed  cost  estimates  for  certification  of  new 
vehicles  and  adequately  budget  for  the  associated  risks.  NASA  is  in  the  process  of 
estimating  costs  to  certify  the  Falcon  9  and  will  budget  accordingly.  Taurus  II  is  not 
currently  included  in  the  NASA  Launch  Services  contract;  however,  NASA  will  follow 
the  same  process  in  the  event  that  it  is  added.  In  addition,  NASA  is  working  to  resolve 
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the  inherent  conflict  between  the  desire  to  minimize  technical  risk  by  identifying  the 
launch  vehicle  as  early  as  possible  (before  Preliminary  Design  Review)  and  the  desire  to 
minimize  programmatic  risk  by  not  committing  to  purchase  a  launch  vehicle  prior  to 
mission  confirmation  (at  Key  Decision  Point-C,  Post-Preliminary  Design  Review). 

NASA  is  concerned  that  technical  corrections  to  the  Project  Two-Page  Summaries 
that  were  provided  in  late  2010  were  not  fully  accepted  by  the  GAO  in  the  draft  report. 
Many  of  these  comments  have  been  resubmitted  for  GAO’s  consideration.  NASA  will 
work  with  the  GAO  team  to  better  understand  why  specific  corrections  were  not  accepted 
and  to  better  explain  our  issues  if  necessary.  In  addition,  NASA  understands  that  the 
GAO's  work  was  completed  in  the  fall  of  2010  and  is  concerned  that  the  assessment, 
therefore,  does  not  recognize  the  significant  progress  the  Agency  has  made  since  then. 

NASA  will  continue  to  follow  through  with  our  new  policies  and  management 
attention  on  cost  and  schedule  growth  in  the  coming  year.  We  are  committed  to 
continuous  improvement  in  order  to  explore  and  utilize  space  in  an  affordable  way  for  the 
benefit  of  the  Nation.  To  this  end,  we  look  forward  to  continuing  to  work  with  the  GAO 
to  measure  and  improve  our  performance. 

Thank  you  for  the  opportunity  to  comment  on  this  draft  report.  If  you  have  any 
questions  or  require  additional  information,  please  contact  Katie  Gallagher  at  (202)  358- 


2185. 


Sincerely, 


Lori  B.  Garver 
Deputy  Administrator 
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Our  objectives  were  to  report  on  the  status  and  challenges  faced  by 
NASA  systems  with  life-cycle  costs  of  $250  million  or  more  and  to  discuss 
broader  trends  faced  by  the  agency  in  its  management  of  acquisitions.  In 
conducting  our  work,  we  evaluated  performance  and  identified  challenges 
for  each  of  21  major  projects.  We  summarized  our  assessments  of  each 
individual  project  in  two  components — a  project  profile  and  a  detailed 
discussion  of  project  challenges.  We  did  not  validate  the  cost  and  schedule 
data  provided  by  NASA,  however,  we  took  appropriate  steps  to  address 
data  reliability.  Specifically,  we  confirmed  the  accuracy  of  NASA-generated 
data  with  multiple  sources  within  NASA  and,  in  some  cases,  with  external 
sources.  Additionally,  we  corroborated  data  provided  to  us  with  published 
documentation.  We  determined  that  the  data  provided  by  NASA  project 
offices  were  sufficiently  reliable  for  our  engagement  purposes. 

We  developed  a  standardized  data  collection  instrument  (DCI)  that  was 
completed  by  each  project  office.  Through  the  DCI,  we  gathered  basic 
information  about  projects  as  well  as  current  and  projected  development 
activities  for  those  projects.  The  cost  and  schedule  data  estimates  that 
NASA  provided  were  the  most  recent  updates  as  of  February  2011; 
performance  data  that  NASA  provided  were  also  the  most  recent  updates 
as  of  September  2010.  At  the  time  we  collected  the  data,  8  of  the  21  projects 
were  in  the  formulation  phase.  Three  of  these  8  projects — MAVEN,  LADEE, 
and  OCO-2 — were  confirmed  and  entered  the  implementation  phase 
late  in  2010.  To  further  understand  performance  issues,  we  talked  with 
officials  from  each  project  office  and  NASA’s  Office  of  the  Chief  Financial 
Officer  (OCFO)  Strategic  Investments  Division  (SID).  We  also  collected 
cost  and  schedule  data  for  projects  in  operations  that  we  had  reviewed  in 
prior  reports  for  historical  purposes.  These  projects  were  DAWN,  GLAST, 
Herschel,  Kepler,  LRO,  OCO,  SDO,  and  WISE. 

The  information  collected  from  each  project  office,  Mission  Directorate, 
and  OCFO/SID  were  summarized  in  a  2-page  report  format  providing  a 
project  overview;  key  cost,  contract,  and  schedule  data;  and  a  discussion  of 
the  challenges  associated  with  the  deviation  from  relevant  indicators  from 
best  practice  standards.  The  aggregate  measures  and  averages  calculated 
were  analyzed  for  meaningful  relationships,  e.g.,  relationship  between  cost 
growth  and  schedule  slippage  and  knowledge  maturity  attained  both  at 
critical  milestones  and  through  the  various  stages  of  the  project  life  cycle. 
Cost  growth  averages  used  in  this  report  are  weighted  averages  and  should 
not  be  used  as  a  point  of  comparison  to  previous  reports  where  weighted 
averages  were  not  used.  We  identified  cost  and/or  schedule  growth  as 
significant  where,  in  either  case,  a  project’s  cost  and/or  its  schedule  baseline 
exceeded  the  thresholds  that  trigger  reporting  to  the  Congress. 


Page  86 


GAO-1 1-239SP  Assessment  of  Selected  Large-Scale  Projects 


Appendix  II:  Objectives,  Scope,  and 
Methodology 


To  supplement  our  analysis,  we  relied  on  GAO’s  work  over  past  years 
examining  acquisition  issues  across  multiple  agencies.  These  reports 
cover  such  issues  as  contracting,  program  management,  acquisition  policy, 
and  estimating  cost.  GAO  also  has  an  extensive  body  of  work  related  to 
challenges  NASA  has  faced  with  specific  system  acquisitions,  financial 
management,  and  cost  estimating.  This  work  provided  the  context  and  basis 
for  large  parts  of  the  general  observations  we  made  about  the  projects  we 
reviewed.  Additionally,  the  discussions  with  the  individual  NASA  projects 
helped  us  identify  further  challenges  faced  by  the  projects.  Together,  the 
past  work  and  additional  discussions  contributed  to  our  development  of 
a  short  list  of  challenges  discussed  for  each  project.  The  challenges  we 
identified  and  discussed  do  not  represent  an  exhaustive  or  exclusive  list. 
They  are  subject  to  change  and  evolution  as  GAO  continues  this  annual 
assessment  in  future  years.  The  challenges,  indicated  as  “issues,”  are  based 
on  our  definitions  and  assessments,  not  that  of  NASA. 

Our  work  was  performed  primarily  at  NASA  headquarters  in  Washington, 
D.C.  In  addition,  we  visited  NASA’s  Marshall  Space  Flight  Center  in 
Huntsville,  Alabama,  and  Goddard  Space  Flight  Center  in  Greenbelt, 
Maryland,  to  discuss  individual  projects.  We  also  met  with  representatives 
from  NASA’s  Jet  Propulsion  Lab  in  Pasadena,  California,  and  a  contractor 
involved  with  several  projects,  Orbital  Science  Corporation.  In  addition, 
we  interviewed  officials  at  Johnson  Space  Center  in  Houston,  Texas;  Ames 
Research  Center  at  Moffitt  Field  in  California;  and  Dryden  Flight  Research 
Center  at  Edwards  Air  Force  Base  in  California. 


Data  Limitations 


NASA  only  provided  specific  cost  and  schedule  estimates  for  16  of  the  21 
projects  in  our  review.  NASA  provided  internal  preliminary  estimated  total 
(life-cycle)  cost  ranges  and  associated  schedules  for  three  of  the  projects 
that  had  not  yet  entered  implementation,  from  key  decision  point  B 
(KDP-B),  solely  for  informational  purposes.19  We  did  not  receive  cost 
estimates  or  ranges  for  two  projects — Ice,  Cloud,  and  Land  Elevation 
Satellite-2  and  Solar  Probe  Plus — since  these  projects  had  not  yet  reached 
their  KDP-B,  the  point  in  the  acquisition  life  cycle  where  a  preliminary 
life-cycle  cost  estimate  would  normally  be  developed.  We  did  receive 
preliminary  scheduled  launch  dates  for  these  two  projects.  NASA  formally 
establishes  cost  and  schedule  baselines,  committing  itself  to  cost  and 
schedule  targets  for  a  project  with  a  specific  and  aligned  set  of  planned 
mission  objectives,  at  key  decision  point  C  (KDP-C),  which  follows  a  non¬ 
advocate  review  (NAR)  and  preliminary  design  review  (PDR).  KDP-C 
reflects  the  life-cycle  point  where  NASA  approves  a  project  to  leave  the 


19  These  missions  include  Ares  I,  Soil  Moisture  Active  and  Passive,  and  Orion. 
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formulation  phase  and  enter  into  the  implementation  phase.  NASA 
explained  that  preliminary  estimates  are  generated  for  internal  planning  and 
fiscal  year  budgeting  purposes  at  KDP-B,  which  occurs  midstream  in  the 
formulation  phase,  and  hence,  are  not  considered  a  formal  commitment  by 
the  agency  on  cost  and  schedule  for  the  mission  deliverables.  NASA  officials 
contend  that  because  of  changes  that  occur  to  a  project’s  scope  and 
technologies  between  KDP-B  and  KDP-C,  estimates  of  project  cost  and 
schedule  can  change  significantly  heading  toward  KDP-C. 

We  requested  earned  value  management  data  for  the  21  projects  and 
received  data  on  1 1  of  them.  However,  this  information  was  received  late  in 
our  review,  and  as  a  result,  we  were  unable  to  conduct  a  detailed  analysis  of 
the  earned  value  data. 

We  also  requested  independent  cost  estimates  and  Joint  Cost  and  Schedule 
Confidence  Levels  (JCL)  for  the  projects  that  completed  them.  We  received 
independent  cost  estimates  for  12  of  the  projects  in  our  review  and  for 
6  projects  that  have  launched  since  our  last  review.  In  most  cases,  we 
received  independent  cost  estimates  conducted  at  the  center  level  by  the 
projects,  along  with  estimates  by  the  Aerospace  Corporation  and/or  by 
NASA’s  Independent  Program  Assessment  Office.  We  received  JCL  analyses 
from  three  of  the  five  projects  that  have  completed  their  JCLs.  However, 
this  information  was  incomplete  and  received  late  in  our  review,  and  as  a 
result,  we  were  unable  to  conduct  a  thorough  analysis  of  the  data. 

Project  Profile  Information  This  section  of  the  2-page  assessment  outlines  the  essentials  of  the  project, 

on  Each  Individual  2-Page  its  cost  and  schedule  performance,  and  its  summary.  Project  essentials 

Assessment  reflect  pertinent  information  about  each  project,  including,  where 

applicable,  the  major  contractors  and  partners  involved  in  the  project. 

These  organizations  have  primary  responsibility  over  a  major  segment  of 
the  project  or,  in  some  cases,  the  entire  project. 

Project  performance  is  depicted  according  to  cost  and  schedule  changes  in 
the  various  stages  of  the  project  life  cycle.  To  assess  the  cost  and  schedule 
changes  of  each  project  we  obtained  data  directly  from  NASA  OCFO/SID 
and  from  NASA’s  Integrated  Budget  and  Performance  documents.  For 
systems  in  implementation,  we  compared  the  latest  available  information 
with  the  cost  and  schedule  baseline  estimates  for  each  project. 
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Project  Challenges 
Discussion  on  Each 
Individual  2-Page 
Assessment 


All  cost  information  is  presented  in  nominal  “then  year”  dollars  for 
consistency  with  budget  data.20  Baseline  costs  are  adjusted  to  reflect  the 
cost  accounting  structure  in  NASA’s  fiscal  year  2009  budget  estimates.  For 
the  fiscal  year  2009  budget  request,  NASA  changed  its  accounting  practices 
from  full-cost  accounting  to  reporting  only  direct  costs  at  the  project  level. 
The  schedule  assessment  is  based  on  acquisition  cycle  time,  which  is 
defined  as  the  number  of  months  between  the  project  start,  or  formulation 
start,  and  projected  or  actual  launch  date.21  Formulation  start  generally 
refers  to  the  initiation  of  a  project;  NASA  refers  to  project  start  as  key 
decision  point  A,  or  the  beginning  of  the  formulation  phase.  The  preliminary 
design  review  typically  occurs  toward  the  end  of  the  formulation  phase, 
followed  by  a  confirmation  review  process,  referred  to  as  key  decision 
point  C,  which  allows  the  project  to  move  into  the  implementation  phase. 
The  critical  design  review  is  generally  held  during  the  latter  half  of  the  final 
design  and  fabrication  phase  of  implementation  and  demonstrates  that  the 
maturity  of  the  design  is  appropriate  to  support  proceeding  with  full-scale 
fabrication,  assembly,  integration,  and  test.  Launch  readiness  is  determined 
through  a  launch  readiness  review  that  verifies  that  the  launch  system 
and  spacecraft/payloads  are  ready  for  launch.  The  implementation  phase 
includes  the  operations  of  the  mission  and  concludes  with  project  disposal. 

We  assessed  the  extent  to  which  NASA  projects  exceeded  their  cost  and 
schedule  baselines.  To  do  this,  we  compared  previously  established  project 
baseline  cost  and  schedule  estimates  with  the  current  cost  and  schedule 
data  reported  by  the  project  office  in  February  2011. 


To  assess  the  project  challenges  for  each  project,  we  submitted  a  data 
collection  instrument  to  each  project  office.  In  the  data  collection 
instrument,  we  requested  information  on  the  maturity  of  critical  and 
heritage  technologies,  number  of  releasable  design  drawings  at  project 
milestones,  and  project  contractors  and  partnerships.  We  also  held 
interviews  with  representatives  from  each  of  the  projects  to  discuss  the 
information  on  the  data  collection  instrument.  These  discussions  led  to 
identification  of  further  challenges  faced  by  NASA  projects.  The  eight 
challenges  we  identified  were  largely  apparent  in  the  projects  that  had 
entered  the  implementation  phase;  however,  there  were  instances  where 


20  Because  of  changes  in  NASA’s  accounting  structure,  its  historical  cost  data  are  relatively 
inconsistent.  As  such,  we  used  “then-year”  dollars  to  report  data  consistent  with  the  data 
NASA  reported  to  us. 

21  Some  projects  reported  that  their  spacecraft  would  be  ready  for  launch  sooner  than  the 
date  that  the  launch  authority  could  provide  actual  launch  services.  In  these  cases,  we 
used  the  actual  launch  date  for  our  analysis  rather  than  the  date  that  the  project  reported 
readiness. 
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these  challenges  were  identified  in  projects  in  the  formulation  phase.  We 
then  reviewed  pertinent  project  documentation — such  as  the  project  plan, 
schedule,  risk  assessments,  and  major  project  reviews — to  corroborate  any 
testimonial  evidence  we  received  in  the  interviews. 

To  assess  issues  with  technology,  we  asked  project  officials  to  provide  the 
technology  readiness  levels  of  each  of  the  project’s  critical  technologies 
at  various  stages  of  project  development.  Originally  developed  by  NASA, 
technology  readiness  levels  are  measured  on  a  scale  of  one  to  nine, 
beginning  with  paper  studies  of  a  technology’s  feasibility  and  culminating 
with  a  technology  fully  integrated  into  a  completed  product.  (See  appendix 
III  for  the  definitions  of  technology  readiness  levels.)  In  most  cases,  we 
did  not  validate  the  project  offices’  selection  of  critical  technologies  or  the 
determination  of  the  demonstrated  level  of  maturity.  However,  we  sought  to 
clarify  the  technology  readiness  levels  in  those  cases  where  the  information 
provided  raised  concerns,  such  as  where  a  critical  technology  was  reported 
as  immature  late  in  the  project  development  cycle.  Additionally,  we  asked 
project  officials  to  explain  the  environments  in  which  technologies  were 
tested. 

Our  best  practices  work  has  shown  that  a  technology  readiness  level  of 
6 — demonstrating  a  technology  as  a  fully  integrated  prototype  in  a  relevant 
environment — is  the  level  of  maturity  needed  to  minimize  risks  for  space 
systems  entering  product  development.  In  our  assessment,  the  technologies 
that  have  reached  technology  readiness  level  6  are  referred  to  as  fully 
mature  because  of  the  difficulty  of  achieving  technology  readiness  level  7, 
which  is  demonstrating  maturity  in  an  operational  environment — space. 
Projects  with  critical  technologies  that  did  not  achieve  maturity  by  the 
preliminary  design  review  were  assessed  as  having  a  technology  issues 
project  challenge.  We  did  not  assess  technology  maturity  for  those  projects 
which  had  not  yet  reached  the  preliminary  design  review  at  the  time  of  this 
assessment.22 

We  also  asked  project  officials  to  assess  the  technology  readiness  level 
of  each  of  the  project’s  heritage  technologies  at  various  stages  of  project 
development.  We  also  interviewed  project  officials  about  the  use  of  heritage 
technologies  in  their  projects.  We  asked  them  what  heritage  technologies 
were  being  used;  what  effort  was  needed  to  modify  the  form,  fit,  and 


22  According  to  NASA  officials,  projects  that  were  in  formulation  at  the  time  of  the  agency’s 
2007  revision  of  its  project  management  policy  are  required  to  comply  with  that  policy. 
Projects  that  had  already  entered  implementation  at  the  time  of  the  revision  were  directed 
to  implement  those  requirements  that  would  not  adversely  affect  the  project’s  cost  and 
schedule  baselines. 
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function  of  the  technology  for  use  in  the  new  system;  whether  the  project 
encountered  any  problems  in  modifying  the  technology;  and  whether  the 
project  considered  the  heritage  technology  as  a  risk  to  the  project.  Heritage 
technologies  were  not  considered  critical  technologies  by  several  of  the 
projects  we  reviewed.  Based  on  our  interviews,  review  of  data  from  the 
data  collection  instruments,  and  previous  GAO  work  on  space  systems, 
we  determined  whether  these  technology  issues  were  a  challenge  for  a 
particular  project. 

To  assess  issues  with  design,  we  asked  project  officials  to  provide  the 
percentage  of  engineering  drawings  completed  or  projected  for  completion 
by  the  preliminary  and  critical  design  reviews  and  as  of  our  current 
assessment.23  In  most  cases,  we  did  not  verify  or  validate  the  percentage  of 
engineering  drawings  provided  by  the  project  office.  However,  we  collected 
the  project  offices’  rationale  for  cases  where  it  appeared  that  only  a  small 
number  of  drawings  were  completed  by  the  time  of  the  design  reviews 
or  where  the  project  office  reported  significant  growth  in  the  number  of 
drawings  released  after  CDR.  In  accordance  with  GAO’s  best  practices, 
projects  were  assessed  as  having  achieved  design  stability  if  they  had  at 
least  90  percent  of  projected  drawings  releasable  by  the  critical  design 
review.  Projects  that  had  not  met  this  metric  were  determined  to  have 
a  design  stability  project  challenge.  Though  some  projects  used  other 
methods  to  assess  design  stability,  such  as  computer  and  engineering 
models  and  analyses,  we  did  not  assess  the  effectiveness  of  these  other 
methods.  We  did  not  assess  design  stability  for  those  projects  that  had  not 
yet  reached  the  critical  design  review  at  the  time  of  this  assessment. 

To  assess  issues  with  funding,  we  interviewed  officials  from  NASA’s  OCFO/ 
SID  and  NASA  project  officials,  and  also  relied  upon  past  interviews  with 
project  contractors  about  the  stability  of  funding  throughout  the  project 
life  cycle.  In  addition,  NASA  received  an  appropriation  from  the  American 
Recovery  and  Reinvestment  Act  of  2009  (ARRA).  NASA  provided  a  record 
of  projects  involved  in  our  review  that  received  ARRA  funds  and  reported 
the  amount  of  ARRA  funds  a  project  received  in  the  cost  tab  of  the  data 
collection  instrument.  We  also  asked  project  and  Mission  Directorate 
officials  to  discuss  how  these  funds  were  used.  Funding  issues  were 
considered  a  challenge  if  officials  indicated  that  project  funding  had  been 
interrupted  or  delayed  resulting  in  an  impact  to  the  cost,  schedule,  or 


23  In  our  calculation  for  percentage  of  total  number  of  drawings  project  for  release,  we  used 
the  number  of  drawings  released  at  critical  design  review  as  a  fraction  of  the  total  number  of 
drawings  projected,  including  where  a  growth  in  drawings  occurred.  So,  the  denominator  in 
the  calculation  may  have  been  larger  than  what  was  projected  at  the  critical  design  review. 
We  believe  that  this  more  accurately  reflected  the  design  stability  of  the  project. 
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performance  of  the  project;  if  the  project  received  ARRA  funding;  or  if 
project  officials  indicated  that  the  project  budgets  do  not  have  sufficient 
funding  in  certain  years  based  on  the  work  expected  to  be  accomplished. 

We  corroborated  the  funding  changes  and  reasons  with  budget  documents 
when  available. 

To  assess  issues  with  launch,  we  interviewed  NASA  Launch  Services  and 
project  officials.  We  also  interviewed  contractor  representatives  from 
Orbital  Sciences  Corporation  to  discuss  the  launch  failure  of  the  OCO-1 
mission  in  2009  and  the  return  to  flight  process  for  the  Taurus  XL  for  the 
Glory  and  OCO-2  missions.  Launch  issues  were  considered  a  challenge  if, 
after  establishing  a  firm  launch  date,  a  project  had  difficulty  rescheduling 
its  launch  date  because  the  project  was  not  ready;  if  the  project  could  be 
affected  by  another  project  slipping  its  launch;  or  if  there  were  launch 
vehicle  fleet  issues.  In  addition,  we  assessed  the  status  of  launch  vehicle 
selection  for  projects  in  formulation  and  considered  it  a  challenge  if  the 
proposed  timing  for  the  launch  vehicle  selection  date  falls  after  Preliminary 
Design  Review  due  the  availability  of  certified  medium  class  launch 
vehicles. 

To  assess  issues  with  contractor  management,  we  interviewed  project 
officials  about  their  interaction  and  experience  with  contractors.  We  also 
interviewed  contractor  representatives  from  Orbital  Sciences  Corporation. 
We  were  informed  about  contractor  performance  problems  pertaining  to 
their  workforce,  the  supplier  base,  and  technical  and  corporate  experience. 
We  assessed  a  project  as  having  this  challenge  if  these  contractor  issues 
caused  the  project  to  experience  a  cost  overrun,  schedule  delay,  or  decrease 
in  mission  capability.  For  projects  that  did  not  have  a  major  contractor,  we 
considered  this  challenge  inapplicable  to  the  project. 

To  assess  issues  with  development  partners,  we  interviewed  NASA  project 
officials  about  their  interaction  with  international  or  domestic  partners 
during  project  development.  Development  partner  issues  were  considered 
a  challenge  for  the  project  if  project  officials  indicated  that  domestic  or 
foreign  partners  were  experiencing  problems  with  project  development 
that  impacted  the  cost,  schedule,  or  performance  of  the  project  for  NASA. 
These  challenges  were  specific  to  the  partner  organization  or  caused  by 
a  contractor  to  that  partner  organization.  For  projects  that  did  not  have 
an  international  or  domestic  development  partner,  we  considered  this 
challenge  not  applicable  to  the  project. 

To  assess  issues  with  parts  quality,  we  submitted  a  data  collection 
instrument  in  conjunction  with  other  on-going  GAO  work  to  all  of  the 
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projects  in  the  implementation  phase  that  were  schedule  to  be  operating 
in  a  space  environment.  In  addition,  we  asked  project  officials  to  identify 
project  components  that  encountered  parts  quality  or  availability  problems 
during  development.  Additionally,  we  asked  project  officials  to  explain  the 
environments  in  which  the  parts  quality  issues  were  discovered  and  any 
implication  on  the  project’s  cost  and  schedule.  We  considered  parts  issues  a 
challenge  if  there  were  actual  or  potential  cost  and/or  schedule  impacts  to 
the  project  as  a  result  of  parts  quality  or  availability,  or  if  the  project  had  to 
take  special  steps  in  order  to  address  parts  issues. 

The  individual  project  offices  were  given  an  opportunity  to  comment  on 
and  provide  technical  clarifications  to  the  2-page  assessments  prior  to 
their  inclusion  in  the  final  product.  We  incorporated  these  comments  as 
appropriate  and  where  sufficient  supporting  documentation  was  provided. 

We  conducted  this  performance  audit  from  March  2010  to  February  2011  in 
accordance  with  generally  accepted  government  auditing  standards.  Those 
standards  require  that  we  plan  and  perform  the  audit  to  obtain  sufficient, 
appropriate  evidence  to  provide  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives.  We  believe  that  the  evidence 
obtained  provides  a  reasonable  basis  for  our  findings  and  conclusions  based 
on  our  audit  objectives. 
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Technology  readiness  level 

Discription 

Hardware 

Demonstration  Environment 

1 .  Basic  principles  observed 
and  reported. 

Lowest  level  of  technology 
readiness.  Scientific  research 
begins  to  be  translated  into  applied 
research  and  development. 

Examples  might  include  paper 
studies  of  a  technology’s  basic 
properties 

None  (paper  studies  and 
analysis) 

None 

2.  Technology  concept  and/or 
application  formulated. 

Invention  begins.  Once  basic 
principles  are  observed,  practical 
applications  can  be  invented.  The 
application  is  speculative  and  there 
is  no  proof  or  detailed  analysis  to 
support  the  assumption.  Examples 
are  still  limited  to  paper  studies. 

None  (paper  studies  and 
analysis) 

None 

3.  Analytical  and  experimental 
critical  function  and / 
or  characteristic  proof  of 
concept. 

Active  research  and  development 
is  initiated.  This  includes  analytical 
studies  and  laboratory  studies 
to  physically  validate  analytical 
predictions  of  separate  elements 
of  the  technology.  Examples 
include  components  that  are  not  yet 
integrated  or  representative. 

Analytical  studies 
and  demonstration  of 
nonscale  individual 
components  (pieces  of 
subsystem). 

Lab 

4.  Component  and/or 
breadboard.  Validation  in 
laboratory  environment. 

Basic  technological  components 
are  integrated  to  establish  that  the 
pieces  will  work  together.  This  is 
relatively  “low  fidelity”  compared 
to  the  eventual  system.  Examples 
include  integration  of  “ad  hoc” 
hardware  in  a  laboratory. 

Low  fidelity  breadboard. 
Integration  of  nonscale 
components  to  show 
pieces  will  work 
together.  Not  fully 
functional  or  form  or 
fit  but  representative 
of  technically  feasible 
approach  suitable  for 
flight  articles. 

Lab 

5.  Component  and/or 
breadboard  validation  in 
relevant  environment. 

Fidelity  of  breadboard  technology 
increases  significantly.  The  basic 
technological  components  are 
integrated  with  reasonably  realistic 
supporting  elements  so  that  the 
technology  can  be  tested  in  a 
simulated  environment.  Examples 
include  “high  fidelity”  laboratory 
integration  of  components. 

High  fidelity  breadboard. 
Functionally  equivalent 
but  not  necessarily 
form  and/or  fit  (size 
weight,  materials,  etc). 
Should  be  approaching 
appropriate  scale.  May 
include  integration  of 
several  components 
with  reasonably 
realistic  support 
elements/subsystems 
to  demonstrate 
functionality. 

Lab  demonstrating  functionality 
but  not  form  and  fit.  May  include 
flight  demonstrating  breadboard 
in  surrogate  aircraft.  Technology 
ready  for  detailed  design  studies. 
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Technology  readiness  level 

Discription 

Hardware 

Demonstration  Environment 

6.  System/subsystem  model 
or  prototype  demonstration  in 
a  relevant  environment. 

Representative  model  or  prototype 
system,  which  is  well  beyond  the 
breadboard  tested  forTRL  5,  is 
tested  in  a  relevant  environment. 
Represents  a  major  step  up  in 
a  technology’s  demonstrated 
readiness.  Examples  include 
testing  a  prototype  in  a  high  fidelity 
laboratory  environment  or  in 
simulated  realistic  environment. 

Prototype. 

Should  be  very  close 
to  form,  fit  and  function. 
Probably  includes  the 
integration  of  many 
new  components  and 
realistic  supporting 
elements/subsystems  if 
needed  to  demonstrate 
full  functionality  of  the 
subsystem. 

High-fidelity  lab  demonstration 
or  limited/restricted  flight 
demonstration  for  a  relevant 
environment.  Integration  of 
technology  is  well  defined. 

7.  System  prototype 
demonstration  in  an  realistic 
environment. 

Prototype  near  or  at  planned 
operational  system.  Represents  a 
major  step  up  from  TRL  6,  requiring 
the  demonstration  of  an  actual 
system  prototype  in  a  realistic 
environment,  such  as  in  an  aircraft, 
vehicle  or  space.  Examples  include 
testing  the  prototype  in  a  test  bed 
aircraft. 

Prototype.  Should  be 
form,  fit  and  function 
integrated  with  other 
key  supporting 
elements/subsystems 
to  demonstrate 
full  functionality  of 
subsystem. 

Flight  demonstration  in 
representative  realistic  environment 
such  as  flying  test  bed  or 
demonstrator  aircraft. 

Technology  is  well  substantiated 
with  test  data. 

8.  Actual  system  completed 
and  “flight  qualified"  through 
test  and  demonstration. 

Technology  has  been  proven  to 
work  in  its  final  form  and  under 
expected  conditions.  In  almost  all 
cases,  this  TRL  represents  the 
end  of  true  system  development. 
Examples  include  developmental 
test  and  evaluation  of  the  system 
in  its  intended  weapon  system 
to  determine  if  it  meets  design 
specifications. 

Flight  qualified  hardware 

Development  Test  and  Evaluation 
(DT&E)  in  the  actual  system 
application 

9.  Actual  system  “flight 
proven”  through  successful 
mission  operations. 

Actual  application  of  the  technology 
in  its  final  form  and  under 
mission  conditions,  such  as  those 
encountered  in  operational  test 
and  evaluation.  In  almost  all 
cases,  this  is  the  end  of  the  last 
“bug  fixing"  aspects  of  true  system 
development.  Examples  include 
using  the  system  under  operational 
mission  conditions. 

Actual  system  in  final 
form 

Operational  Test  and  Evaluation 
(OT&E)  in  operational  mission 
conditions 

Source:  GAO  and  its  analysis  of  NASA  data. 
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